Overview

This arti­cle pro­vides some infor­ma­tion we feel will be help­ful in plan­ning a statewide SIF imple­men­ta­tion in Aus­tralia based on some of the projects we've worked on there and things we've learned while imple­ment­ing those projects.

NOTE: For a basic intro­duc­tion to SIF, please see: Intro­duc­tion to SIF

Most of the Aus­tralian states already have juris­dic­tion-wide Stu­dent Man­age­ment Sys­tems (SMS) that store detailed infor­ma­tion for every stu­dent, teacher and par­ent in the juris­dic­tion. These sys­tems were, for the most, part devel­oped in-house and tai­lored for the spe­cif­ic needs for the jurisdiction.

In many places, each school will have a SMS installed local­ly and the data from that sys­tem will auto­mat­i­cal­ly be repli­cat­ed to update the cen­tral, juris­dic­tion locat­ed sys­tem. This is in many places, but not in all, for some juris­dic­tions have and some are mov­ing to a mod­el where the cen­tral mod­el is host­ed and the schools all access the com­mon sys­tem remote­ly. Regard­less of the mod­el, it is high­ly like­ly that there will be some sys­tems that will con­tin­ue to be installed and main­tained local­ly in many, most or all of the jurisdictions.

SIF Architecture

In this sec­tion, we are going to build an archi­tec­ture step at a time, each step resolv­ing some of the issues com­mon to a typ­i­cal Aus­tralian imple­men­ta­tion. In the fol­low­ing dia­grams, cer­tain types of box­es will be used to rep­re­sent dif­fer­ent types of entities:

	Shapes filled in blue rep­re­sent appli­ca­tions that have either been built by the juris­dic­tion or, in some oth­er way, are not part of the SIF infrastructure.
	Shapes filled in green rep­re­sent soft­ware com­po­nents that are part of the SIF infra­struc­ture that work with an appli­ca­tion to adapt them to work with the SIF Infrastructure.
	Shapes filled in red rep­re­sent inde­pen­dent SIF infra­struc­ture components.

State (Jurisdiction) Level 

At the juris­dic­tion lev­el, data needs to move from the con­sol­i­dat­ed stu­dent infor­ma­tion sys­tem in (near) real time to the Data Ware­house, Busi­ness Intel­li­gence and Report­ing Appli­ca­tions. Since its scope is juris­dic­tion-wide and since the scope of the data need­ed by these appli­ca­tions is also state-wide, the sim­plest archi­tec­ture might be to move all the data through a sin­gle SIF zone as illus­trat­ed in the fol­low­ing diagram:

Although this may appear at first glance like a good and log­i­cal imple­men­ta­tion, it doesn't work very well in prac­tice, espe­cial­ly for large imple­men­ta­tions. For more infor­ma­tion on why, see: SIF_architecture_choices_and_performance_impacts.

Since sub­scrib­ing appli­ca­tions (receivers of data) gen­er­al­ly pre­fer data to be received seri­al­ly, there is a rule in the SIF spec­i­fi­ca­tion that says that the data must be received in the same order that it was sent, This means that there is a sin­gle stream of data being sent between sender and receiv­er. When all data is com­bined through that same stream, it is much less effi­cient that if the data were trans­ferred over many streams.

The oth­er issue aris­es: What if the schools want to make use of the SIF facil­i­ties and this data for automat­ing their appli­ca­tions? If there were a sin­gle, con­sol­i­dat­ed zone, this would not be possible.

For one rea­son, each of these appli­ca­tions would have access to all infor­ma­tion about every stu­dent in every school in the state. So if the appli­ca­tion did a "request all stu­dents", it would receive every stu­dent record in the state.

For anoth­er. this would end up being an unmanagable cen­tral zone hav­ing sub­scribers from every school in the jurisdiction.

It is for this need that Visu­al Soft­ware cre­at­ed their Aus­tralian Ver­sion of Envoy. In short, what it does is to split the con­sol­i­dat­ed State-Lev­el zone into small­er, inde­pen­dent school-lev­el zones in real time, auto­mat­i­cal­ly. To find more infor­ma­tion about this, see: SIF_Zone_Partitioning_Using_Managed_Virtual_Zones.

School Zone Model

NOTE: This mod­el is some­thing unique to Visu­al Soft­ware because of our Envoy prod­uct (our com­peti­tors are still claim­ing that it is impos­si­ble to do).

See­ing the prob­lems with the sin­gle zone mod­el in larg­er imple­men­ta­tions, Visu­al Soft­ware cre­at­ed our Envoy prod­uct (you can watch it work in the video at the end of the first ref­er­ence above). It takes a sin­gle zone imple­men­ta­tion as described above and breaks it into some­thing that looks like this:

The con­sol­i­dat­ed (state lev­el) zone still exists and the con­sol­i­dat­ed provider remains its provider. Envoy, how­ev­er, becomes a sub­scriber in it. It inspects each mes­sage that gets sent to that some and, based on that message's rela­tion­ship to a school, for­wards it to the appro­pri­ate school zone. For some mes­sages, this is sim­ple; for oth­ers it is dif­fi­cult. For still oth­ers, the mes­sage needs to be tem­porar­i­ly staged until enough infor­ma­tion is avail­able to for­ward it.

State-Level Subscribers

State-lev­el sub­scribers may still sub­scribe in the con­sol­i­dat­ed (state-lev­el) zone or they may sub­scribe in each of the school zones inde­pen­dent­ly. From look­ing at the two pic­tures it may appear "bet­ter" to sub­scribe in the one zone, but it is prob­a­bly much more effi­cient to be a sub­scriber in the indi­vid­ual zones. Here's why: When sub­scrib­ing in the sin­gle zone, all data comes through a sin­gle-thread­ed data stream — the SIF agent isn't fed anoth­er mes­sage until it is fin­ished processed pro­cess­ing the one it just received by the ZIS. For an appli­ca­tion like a Data Ware­house, where it isn't con­cerned what order the mes­sages come in, this is not very efficient.

If the appli­ca­tion sub­scribes in each of the zones, it will receive data on that many chan­nels simul­ta­ne­ous­ly (or up to the num­ber allowed by its oper­at­ing sys­tem). This will result in a many-fold through­put improve­ment, which will be huge­ly impor­tant when con­vert­ing and at the end of the school year when doing the roll over to the new year.

School-Level Subscribers

Using this mod­el, there is the pos­si­bil­i­ty of afford­ing this tech­nol­o­gy to schools and host­ing the ser­vice. We have found that a sin­gle, host­ed ZIS can scale to pro­vide these ser­vices not only at the state lev­el, but also to the indi­vid­ual schools as well.