Planning an Australian SIF Implementation
How Can We Help?
Overview
This article provides some information we feel will be helpful in planning a statewide SIF implementation in Australia based on some of the projects we've worked on there and things we've learned while implementing those projects.
NOTE: For a basic introduction to SIF, please see: Introduction to SIF
Most of the Australian states already have jurisdiction-wide Student Management Systems (SMS) that store detailed information for every student, teacher and parent in the jurisdiction. These systems were, for the most, part developed in-house and tailored for the specific needs for the jurisdiction.
In many places, each school will have a SMS installed locally and the data from that system will automatically be replicated to update the central, jurisdiction located system. This is in many places, but not in all, for some jurisdictions have and some are moving to a model where the central model is hosted and the schools all access the common system remotely. Regardless of the model, it is highly likely that there will be some systems that will continue to be installed and maintained locally in many, most or all of the jurisdictions.
SIF Architecture
In this section, we are going to build an architecture step at a time, each step resolving some of the issues common to a typical Australian implementation. In the following diagrams, certain types of boxes will be used to represent different types of entities:
Shapes filled in blue represent applications that have either been built by the jurisdiction or, in some other way, are not part of the SIF infrastructure. | |
Shapes filled in green represent software components that are part of the SIF infrastructure that work with an application to adapt them to work with the SIF Infrastructure. | |
Shapes filled in red represent independent SIF infrastructure components. |
State (Jurisdiction) Level
At the jurisdiction level, data needs to move from the consolidated student information system in (near) real time to the Data Warehouse, Business Intelligence and Reporting Applications. Since its scope is jurisdiction-wide and since the scope of the data needed by these applications is also state-wide, the simplest architecture might be to move all the data through a single SIF zone as illustrated in the following diagram:
Although this may appear at first glance like a good and logical implementation, it doesn't work very well in practice, especially for large implementations. For more information on why, see: SIF_architecture_choices_and_performance_impacts.
Since subscribing applications (receivers of data) generally prefer data to be received serially, there is a rule in the SIF specification that says that the data must be received in the same order that it was sent, This means that there is a single stream of data being sent between sender and receiver. When all data is combined through that same stream, it is much less efficient that if the data were transferred over many streams.
The other issue arises: What if the schools want to make use of the SIF facilities and this data for automating their applications? If there were a single, consolidated zone, this would not be possible.
For one reason, each of these applications would have access to all information about every student in every school in the state. So if the application did a "request all students", it would receive every student record in the state.
For another. this would end up being an unmanagable central zone having subscribers from every school in the jurisdiction.
It is for this need that Visual Software created their Australian Version of Envoy. In short, what it does is to split the consolidated State-Level zone into smaller, independent school-level zones in real time, automatically. To find more information about this, see: SIF_Zone_Partitioning_Using_Managed_Virtual_Zones.
School Zone Model
NOTE: This model is something unique to Visual Software because of our Envoy product (our competitors are still claiming that it is impossible to do).
Seeing the problems with the single zone model in larger implementations, Visual Software created our Envoy product (you can watch it work in the video at the end of the first reference above). It takes a single zone implementation as described above and breaks it into something that looks like this:
The consolidated (state level) zone still exists and the consolidated provider remains its provider. Envoy, however, becomes a subscriber in it. It inspects each message that gets sent to that some and, based on that message's relationship to a school, forwards it to the appropriate school zone. For some messages, this is simple; for others it is difficult. For still others, the message needs to be temporarily staged until enough information is available to forward it.
State-Level Subscribers
State-level subscribers may still subscribe in the consolidated (state-level) zone or they may subscribe in each of the school zones independently. From looking at the two pictures it may appear "better" to subscribe in the one zone, but it is probably much more efficient to be a subscriber in the individual zones. Here's why: When subscribing in the single zone, all data comes through a single-threaded data stream — the SIF agent isn't fed another message until it is finished processed processing the one it just received by the ZIS. For an application like a Data Warehouse, where it isn't concerned what order the messages come in, this is not very efficient.
If the application subscribes in each of the zones, it will receive data on that many channels simultaneously (or up to the number allowed by its operating system). This will result in a many-fold throughput improvement, which will be hugely important when converting and at the end of the school year when doing the roll over to the new year.
School-Level Subscribers
Using this model, there is the possibility of affording this technology to schools and hosting the service. We have found that a single, hosted ZIS can scale to provide these services not only at the state level, but also to the individual schools as well.