Envoy™ Virtual Zone Manager
Envoy is a multi-faced SIF agent that solves a number of challenges encountered by larger organizations implementing a SIF architecture to be shared by a large number of schools and/or local authorities. These challenges include:
- Accessing consolidated data from multiple schools as if it came from a single source with records from many school-level zones matched by student, teacher, contact, and school;
- Protecting the privacy of student data at different levels of integration, considering that all information accessible at the school level may not be legally accessible in other places;
- Working with differing levels of technical sophistication at the school level, from simple spreadsheets to the latest generation student management systems from a mix of suppliers
Envoy is a SIF Virtual Zone Manager that allows its users to define virtual zones as groups of schools as small as a single school and as large as all of the schools in the deployment. SIF-enabled applications that subscribe in one of these virtual zones see the data appear as if there were a single provider for each object.
If a Virtual Learning Environment (VLE) application subscribes in the LA Zone, it sees learners from all 30 schools. It also sees teachers, parents, school groups, schedules, attendance records, assessment records and all of the other associated SIF records that were generated by all of the systems that were attached to the 30 school zones.
If any of the learners attend more than one school (they could either be 14–19 or special education learners), then his or her records would have already been pre-matched and the VLE would have only seen one unified set of records for that learner.
Note: When creating the virtual zone, the providers for the SIF object types are not required to be from the same supplier. In fact, they don't need to be directly attached to the same Zone Integration Server (they could be attached to a ZIS in another region). So, in another example, we create a different virtual zone containing the 45 schools that feed a special education school that is located in one of our towns. 37 of these school are in our area, but the rest are in neighboring areas. The only difficulty is that we will need to maintain HTTPS certificates differently when we set up these connections.
By creating virtual zones where they are needed, Envoy allows subscribing applications to get exactly the set of data they need without missing any of the information typically missed by agents in a multi-zone environment. The SIF agents are much less complex, and on average much more reliable.
"Managed Virtual Zones"
Managed Virtual Zones is the architecture behind the product Envoy™. It uses current versions of SIF specifications "as is" to provide a highly-scalable platform for large SIF implementations. We originally developed this architecture to meet the needs of large implementations in the United Kingdom. The objectives of this architecture are:
- …to eliminate the need for SIF agents to manage more than a single zone. This reduces the overall complexity of the system by having the multi-zone intricacies managed by a central authority and by doing so, increasing the reliability of the entire deployment.
- …to reduce the total number of connections, and by doing so, make the overall system easier to manage, simpler to implement and less expensive to maintain.
- …to handle certain transactions that could not be managed before because the information was not available or because a multi-zone agent had no way to know in which zone to make a request.
- …to provide a framework where the decision-making process involving how to match records for ALL SIF objects (and when copies are to be distributed) is consistent, centralized and made through business rules. Furthermore, these rules can be audited and easily changed on a case-by-case basis if necessary. With the multi-zone architecture approach, a certain amount of this gets done in every agent and every agent developer invents his or her own (unaudited) logic for record matching.
Designed for Reliability
The principle behind this is as follows: These two pictures represent the two typical architectures: the top picture represents a SIF Multi-Zone Architecture and the bottom picture represents a SIF Managed Virtual Zone Architecture.
In the top picture the center circle represents a ZIS and the outer circles represent SIF agents (the applications were omitted to make the diagram simpler). In the bottom picture, the center circle represents the Virtual Zone Manager (Envoy) and a ZIS (working together) and the outer circles represent SIF agents as they did in the top picture.
Circles that have the word "Complex" written on them need to:
- …manage multiple sets of SIF object identifiers for each object needed by an application and keep them all straight. If a multi-zone-based Identity Management (IDM) solution is in place, sets of Identity objects will also need to be managed for some of the objects and cross-referenced.
- …match records if a SIF object represents something that (in real life) appears in more than one school (such as a learner that attends more than one school). Even if a multi-zone-based IDM solution is in place, a considerable amount of matching will still need to be done.
- …be careful when publishing data to use the right mixture of RefIds (it may have received several RefIds for the same SIF object)
In the first scenario, the complexity of the overall system increases substantially and the reliability decreases every time you add a new agent to it, especially if that agent publishes anything. Because the new agent will contain new matching code (for those objects not handled by the IDM system), its interaction with all the other agents will need to be tested before being put into production.
In the second scenario, the one piece of software (Envoy) has all the responsibility for matching and combining records for all agents, so it is safe to assume that all record matching will be consistent and that you will be aware beforehand how this matching will be done. In fact, if the rules for matching violate some local policy, they can be changed through Envoy's web interface. This leaves the SIF agents with much less to do and a much higher probability of doing it correctly. Furthermore, if something does go wrong, it is likely that the problem can be traced back with a reasonable amount of effort.
To learn more about this architecture, see Managed Virtual Zones (login may be required).
Designed for Performance
One of the most (if not the most) influential contributors to a high-performance system is a solid, well-thought out design. In a large system such as this where messages are traded about, they move from place to place and the relative speeds of these places are vastly different. Listed (in order from fastest to slowest) are:
- being handled by a processor (an instruction takes about .33 nanoseconds (Intel Core Duo 3GHz))
- being moved around in cache memory (about 1 nanosecond)
- bring moved around in memory (RAM) (about 80 ns)
- being written to or read from a local disk drive (or in a database on the same machine) (about 15 milliseconds)
- being transmitted over a LAN (HTTP) (or being stored in a nearby database) (about 80 milliseconds)
- being transmitted over a LAN (HTTP) where the connection needs to be re-established for every message (much longer on average)
- being transmitted over a WAN (HTTPS) (varies, typically 3 times LAN or worse)
- being transmitted over a WAN (HTTPS) where the connection needs to be re-established for every message
…and we can leave out the others that aren't applicable for SIF infrastructures.
Certain trade-offs are naturally worthwhile or required for various reasons. But when things such as HTTPS are required for business reasons, it is all the more important that the number of messages that get transmitted are kept to a minimum.
Comparing the same two alternatives as above, we consider the traffic generated for the simplest of cases: one learner's demographic information being entered. This learner attends two schools and, in the Multi-Zone Architecture example, the LA is using an Identity Management solution that uses the "Identity" SIF object for matching learner records.
|Message traffic over HTTPS||Message Sent|
The subscribing applications receives two LearnerPersonal and two Identity messages for the newly enrolled learner. The two "Identity" objects would help the SIF agent to know that the two LearnerPersonal records were for the same Learner. The SIF agent for the application would then be responsible for determining which LearnerPersonal record was more trustworthy.
|Envoy||The subscribing application receives one LearnerPersonal message. Nothing else to do.||1|
There are many other performance advantages of Envoy's architecture, some even more dramatic than the 4:1 advantage above. For a more complete explanation, see Scalability.
As implementations grow, administrative demands grow with them. Envoy presents solutions for administrators in a number of areas:
- It cuts down on the total number of connections to manage.
- It provides a simple way to define data protection levels. Whenever a virtual Zone is created, a set of element-level filters can be established for that zone. These become the default filters for all applications attached to that zone. The ZIS element level filtering is then only needed when an application's filtering requirements exceed that of the zone's.
- It provides a mechanism to audit all traffic going through the system in a centralized location.
In the first scenario, a connection is made between the application and each of the zones for the schools.
If the application is supported by a "pull mode" SIF agent, then it will need to send regular messages to each of these zones to ask if there are any messages waiting.
Also, whenever there is any administration that needs to be done, it will need to be done to each of these connections. So, if this application is being shared at an RBC between 2,500 schools, there will be 2,500 connections to administer. This also means 2,500 sets of element level filters that need to be audited, solely for this application.
Each of these three agents will need to directly connect to the school zones. If the agent uses "pull mode", then it will need to send "poll" messages to each of the zones on a regular basis.
As in the example above, if these applications were being shared from an RBC, then there would be 10,000 connections to manage and 10,000 sets of element level filters to manage.
Lastly, add to this the other tiers that would be in a real implementation. For example, applications that would be shared at the Local Authority level, those that would be shared among 14–19 partnerships and those that would be shared among special education schools.
In the Envoy scenario, each of the schools still have a SIF zone (depicted by the blue "School" shapes in this diagram). Any application that needs information from only that school would register in that zone.
If an application is to be shared between all the schools in a community, then a virtual zone would be created for that community and the application would register in the virtual zone.
This virtual zone could be given a set of element level filters that would apply to all applications in the zone (Envoy refers to this as a "blanket-level filter"). The most important things is that the application would have a single connection to virtual zone. If it was a SIF "pull" agent, it would be sending a single poll message to the virtual zone instead of individual poll messages to each of the school zones.
Built to Scale
Because Envoy handles every message passed between any application and any other application and needs to translate most of them, having an ability to scale is essential. In both our Standard and Enterprise Envoy models, we provide options to cluster servers that provide high-availability and high-performance at a low incremental cost. Web servers can be load balanced using Windows Server Network Load Balancing or hardware devices and database servers can be scaled using one of a number of techniques we support.