Implementing SIF Reliability and Privacy

When build­ing a soft­ware prod­uct, there are two approach­es to test­ing it. A while back, I learned that the way we do this at our com­pa­ny is a bit out of the ordi­nary, although I nev­er knew that we were that dif­fer­ent. Let me illus­trate" The typ­i­cal approach to build­ing a soft­ware prod­uct is to design it, write it, and then run it to see if the results match the results you expect­ed when you designed it. In test­ing it, you use test data that some­one else cre­at­ed to the best of his or her ability.

Our approach, on the oth­er hand, has been to design the soft­ware, write it, and then run the soft­ware in "sin­gle step mode" in the devel­op­ment envi­ron­ment and prove that it works cor­rect­ly each step of the way, using test data specif­i­cal­ly designed to find every pos­si­ble type of error by those writ­ing the code (they know every junc­tion in the code). It takes far longer to do this the first time, but in the long run, we end up spend­ing far less time than the "typ­i­cal" approach men­tioned above.

We feel that the best way to test code is to prove that it works instead of test­ing to see if it doesn't work and then going back and patch­ing it up where need­ed. In the typ­i­cal approach, the error is detect­ed by look­ing at the results, notic­ing they do not meet expec­ta­tions, then work­ing back­wards to find the source of the prob­lem. This sit­u­a­tion is illus­trat­ed in the fol­low­ing exam­ple where the source of the prob­lem was only found by exam­in­ing "post-process 6" results.

When we iden­ti­fy the prob­lem in Process 3, we might be tempt­ed to do what­ev­er was need­ed to fix it there. While it might fix the issue found in this par­tic­u­lar test, we may over­look the sub­tle prob­lem in Process 2 that could emerge at some oth­er time. Using the "prove that it works" method, you nev­er make either of these errors in the first place. While still imper­fect, this method ensures that when devel­op­ment is com­plete the code is in sync with the programmer's under­stand­ing of the require­ments. This method can­not com­plete­ly elim­i­nate human fac­tors such as:

1. The pro­gram­mer didn't under­stand the specification/requirements;
2. Defi­cien­cies in tools on which we depend;
3. The specification/requirements were insuf­fi­cient or inaccurate.

Despite these imper­fec­tions, this method con­sis­tent­ly reduces code errors and pro­vides a frame­work for the pro­gram­mer to ver­i­fy his or her under­stand­ing of the specification/requirements is reflect­ed in his or her code.

What does this have to do with SIF?

You might be read­ing this won­der­ing how this could pos­si­bly relate to a SIF imple­men­ta­tion. Take for exam­ple, a typ­i­cal US SIF imple­men­ta­tion (we will NOT be rec­om­mend­ing some­thing like this):

1. Install a Zone Inte­gra­tion Serv­er and con­fig­ure it
2. Install HTTPS cer­tifi­cates as needed
3. Install SIF agent for MIS (or SIS as they call them in the US)
4. Install an agent for a sub­scriber (let's say a VLE)
5. Run the con­ver­sion util­i­ties in the VLE and see if all the data makes it there as it should

In this exam­ple, you would dis­cov­er prob­lems with your SIF imple­men­ta­tion if some teach­ers, par­ents or learn­ers nev­er made it into the VLE.

If there were a prob­lem, you would nev­er quite know where the prob­lem orig­i­nat­ed with­out quite a bit of work (was there a prob­lem with the MIS, with the MIS SIF agent, with the ZIS, with the VLE SIF agent or with the VLE?). Fur­ther­more, you may have pur­chased your MIS from one sup­pli­er, your ZIS from anoth­er and your VLE from still anoth­er. You could also have pur­chased the SIF agents from some­one oth­er than the MIS or VLE suppliers.

What we will be sug­gest­ing in this paper is some­thing anal­o­gous to our soft­ware design approach, where the imple­men­ta­tion is built incre­men­tal­ly and is proven to work prop­er­ly at each junc­ture so that at the end of the process there is a much reduced risk of hav­ing some­thing hap­pen­ing like the above. What we are sug­gest­ing is to not use your sub­scrib­ing appli­ca­tions to "debug" your SIF imple­men­ta­tion "“ there is a bet­ter way.

Note: this doc­u­ment is being writ­ten by an Amer­i­can to a British audi­ence, so Amer­i­can words (as in lawyer vs. bar­ris­ter) will be used with British ter­mi­nol­o­gy (as in Learn­er­Per­son­al vs. Stu­dent­Per­son­al), so you might need to read it with a bot­tle of aspirin handy"¦

Things to Consider Beforehand

Hear­ing about SIF for the first time is excit­ing, espe­cial­ly if you've endured the headaches of export and import, reject­ed batch­es, resub­mis­sion, and all that goes with it. Before we get into con­nect­ing appli­ca­tions and exchang­ing data there are two com­mon­ly over­looked areas that should be con­sid­ered: data qual­i­ty and data privacy. Prop­er­ly address­ing these two issues before data starts mov­ing is crit­i­cal to any suc­cess­ful SIF implementation.

Data Quality

Data Qual­i­ty is an issue that will like­ly catch many by sur­prise. In years past, main­tain­ing "less than clean" data in a school's MIS had far less impact than it will when that system's data becomes the author­i­ty for most of the appli­ca­tions in the school, the local author­i­ty, the RBC and even­tu­al­ly as the basis for report­ing to the DCSF. Before SIF, if some words were typed into a "phone num­ber" field instead of a real phone num­ber (for exam­ple, the word "UNLISTED" was typed where the num­ber should go), that might be accept­able because every­one who used the MIS knew what it meant. But when the MIS is used as the sup­pli­er of data for every­one who wants learn­er data, only the num­ber itself will be accept­able in the phone num­ber field.

Why then could SIF be imple­ment­ed in the US as described ear­li­er? Reject­ed mes­sages were nev­er an issue in the US, because the US SIF schema (the mech­a­nism used to pro­gram­mat­i­cal­ly check the cor­rect­ness of data) val­i­dates very few fields where the UK schema val­i­dates many. In the US, almost any val­ue is passed through by the ZIS with no com­plaints where in the UK, the ZIS will end up reject­ing a much high­er per­cent­age of mes­sages. Is one good and one bad? They're dif­fer­ent. The impor­tant thing is that the prin­ci­ples that work well in one envi­ron­ment may not work so well in the other.

Types of Errors

Errors can be intro­duced in the source data; for the sake of the remain­der of this doc­u­ment, I'll clas­si­fy them as follows:

• Type 1: The dif­fer­ence between the val­i­da­tion that your MIS sys­tem per­forms on input and that which the ZIS does when it con­sid­ers a mes­sage as "accept­able" for trans­mis­sion to anoth­er appli­ca­tion. For exam­ple, an MIS may allow a user to type any­thing for a teacher's Nation­al Insur­ance Num­ber (NINum­ber), but if it isn't exact­ly in the form <2 let­ters fol­lowed by 6 num­bers fol­lowed by 1 let­ter>, the Work­for­cePer­son­al record will not be published.
• Type 2: This is where a data ele­ment is option­al in a MIS, is not typ­i­cal­ly entered, but is required by a sub­scrib­ing appli­ca­tion to work prop­er­ly. For exam­ple, a VLE sub­scribes to a Learn­er­SchoolEn­rol­ment object because it needs to know about the lev­el at which a stu­dent speaks Eng­lish. Specif­i­cal­ly, what it needs from this object is an option­al ele­ment named "Eng­lish­Pro­fi­cien­cy". If deliv­ered, the VLE works as it should. The gap between what the MIS pub­lish­es and what the appli­ca­tion needs rep­re­sent data entry pol­i­cy that needs to be changed and/or data that needs to be cleaned up at the school.
• Type 3: Ele­ments that the MIS does not sup­port at all, the SIF spec­i­fi­ca­tion spec­i­fies as option­al (hence the ZIS allows mes­sages to pass with­out them present), but the sub­scrib­ing appli­ca­tion may not work prop­er­ly with­out them present. This hap­pens more than you might expect. For exam­ple, when an MIS pub­lish­es School­Group objects, the timetabling infor­ma­tion is option­al. There are many sub­scrib­ing appli­ca­tions that may not work prop­er­ly if School­Group objects are sup­plied with­out timetabling information.
• Type 4: Data that meets or doesn't meet local stan­dards "“ these may be stan­dards defined to reflect local­ly or region­al­ly defined poli­cies. Exam­ples may include : every stu­dent must have a birth date that is rea­son­able for his or her NCYear­Group; if a stu­dent is reg­is­tered in "Eng­lish as a Sec­ond Lan­guage" as a course, then he or she shouldn't have Eng­lish as the pri­ma­ry lan­guage. We'll also throw into this cat­e­go­ry oth­er ran­dom but com­mon data entry errors.
• Type 5: Data that may or may not meet nation­al cen­sus stan­dards. Schools and local author­i­ties spend enor­mous time and effort annu­al­ly prepar­ing data for these reports and in most places, spend too much time fix­ing the same errors over and over. If an error can be fixed at the source with the same lev­el of effort (or less) than fix­ing it "the old way", then the sav­ings will start to mul­ti­ply since most of the same data is repeat­ed from report­ing cycle to cycle.

Data Pollution

With SIF in place, the MIS is going to gain new author­i­ty "“ it will offi­cial­ly become the new "dig­i­tal author­i­ty" for learn­er, con­tact and teacher infor­ma­tion at the school. Before SIF it was only respon­si­ble for its own data (unless it gen­er­at­ed extract files that were sent else­where). What this means is that as soon as an error is entered in the MIS it will prop­a­gate through­out the school, and some­times even over­write cor­rect infor­ma­tion in oth­er appli­ca­tions. I like to refer to this as "data pollution.

Errors vs. Unmet Expectations 

Unfor­tu­nate­ly, since these errors orig­i­nate in so many places and are many times they aren't even errors but rather mis­match­es in capa­bil­i­ties. It is advan­ta­geous to be pre­pared for them and to be able to set expec­ta­tions before­hand. Some­times exer­cis­es like this may even lead to a con­clu­sion that the project is not worth doing at all.

For exam­ple, if your only expec­ta­tion of the entire project depends on a sub­scrib­ing appli­ca­tion being able to do some­thing based on infor­ma­tion that your MIS doesn't even store (and nev­er will), then per­haps the project is more than sim­ply buy­ing a ZIS and a few SIF agents.

Data Privacy

Data pri­va­cy is one area where there are many legal dif­fer­ences between the US and the UK that must be under­stood before start­ing a SIF imple­men­ta­tion. This is impor­tant because the SIF spec­i­fi­ca­tion was orig­i­nal­ly designed by peo­ple with a good under­stand­ing of US laws and cus­toms and some­times these designs bump up against UK requirements.

For exam­ple, in the US, the pre­dom­i­nant author­i­ty for pri­va­cy of stu­dent data is the Fam­i­ly Edu­ca­tion­al Rights and Pri­va­cy Act (FERPA ). FERPA doesn't real­ly deal with pass­ing of infor­ma­tion between appli­ca­tions in the same school or between schools in the same com­mu­ni­ty "“ it most­ly address­es when data is moved from dis­trict (like a Local Author­i­ty) to state. In the UK, schools are required to oper­ate under the require­ments of the Data Pro­tec­tion Act (DPA) of 1998, the Edu­ca­tion Act of 1996 and oth­ers which have spe­cif­ic guide­lines that gov­ern the trans­mis­sion of what the DPA refers to as "Edu­ca­tion­al Records" (SCHEDULE 11, Sec­tion 68 (1)(6)), between depart­ments in schools as well as between appli­ca­tions in the same school. For more infor­ma­tion, see Data Pro­tec­tion Good Prac­tice Note .

Giv­en these dif­fer­ences, SIF was designed with­out "fine-tuned secu­ri­ty" when con­nect­ing appli­ca­tions hor­i­zon­tal­ly, because in the US, it was nev­er an issue. The MIS pub­lish­es the infor­ma­tion to all who are lis­ten­ing "“ it is up to the sub­scriber to take what it wants, ignore what is not appro­pri­ate. It relies on trust­ing the integri­ty of the sub­scribers to not do "bad things" with sen­si­tive infor­ma­tion that has been passed to them that they didn't need.

Resolving the Differences

There are real­ly two issues: the exist­ing SIF objects that were designed for the UK using the US "way of think­ing" and "what to do going for­ward". I believe both have answers; the first took some devel­op­ment and the sec­ond may take some changes in the way we design new objects in the future. I'll address the sec­ond one first"¦

Going Forward

This is strict­ly opin­ion, but I believe that peo­ple should con­sid­er data sen­si­tiv­i­ty when design­ing new objects. Design­ers should dif­fer­en­ti­ate sen­si­tive data from non-sen­si­tive data by split­ting it into sep­a­rate objects. In a way, this was done with the Learn­er­Spe­cial­Needs object "“ most, if not all of the infor­ma­tion in this object is sen­si­tive, so SIF object lev­el per­mis­sions should be suf­fi­cient in safe­guard­ing access to this infor­ma­tion. For exam­ple, if the DPA had been con­sid­ered in the orig­i­nal design of Learn­er­Per­son­al, it might have been split into two objects: one con­tain­ing basic stu­dent infor­ma­tion and a sec­ond con­tain­ing infor­ma­tion cov­ered by the DPA, both con­tain­ing the same LearnerPersonalRefId.

Working with Existing Objects

To han­dle the "mixed objects", we've added a fea­ture called "Pri­va­cy­Plus™" to two of our prod­ucts: ZIS­erv­er™ and Envoy™. Pri­va­cy­Plus essen­tial­ly puts a "strain­er" in front of the "fire hose" or, more tech­ni­cal­ly, allows an admin­is­tra­tor to set up a series of fil­ters that deter­mine exact­ly which infor­ma­tion is allowed to go where. For ZIS­erv­er (the Zone Inte­gra­tion Serv­er), the Pri­va­cy­Plus fea­ture can restrict data from being sent to a par­tic­u­lar application.

When the sub­scrib­ing agent is set up in the ZIS, the admin­is­tra­tor can choose which ele­ments in a mes­sage would be fil­tered out before send­ing a mes­sage to this agent. This is use­ful for restrict­ing infor­ma­tion between appli­ca­tions with­in a school. In this exam­ple, the admin­is­tra­tor selects which ele­ments (or group of ele­ments) will not be sent to a sub­scrib­ing appli­ca­tion. When ZIS­erv­er receives a mes­sage for dis­tri­b­u­tion from the provider, it will strip out this infor­ma­tion before pass­ing the infor­ma­tion on to this appli­ca­tion. Since there can be sev­er­al dif­fer­ent "ver­sions" of the same mes­sage being dis­trib­uted, each dif­fer­ent ver­sion will be audit­ed separately.

For Envoy, Pri­va­cy­Plus pro­vides blan­ket-lev­el fil­ter­ing for imple­ment­ing poli­cies con­cern­ing what infor­ma­tion can be trans­mit­ted from one type of orga­ni­za­tion to anoth­er. For exam­ple, if a pol­i­cy states that only the basic learn­er (no med­ical or sen­si­tive per­son­al) infor­ma­tion can be trans­ferred from a school to a local author­i­ty, a set of fil­ters (sim­i­lar to those in ZIS­erv­er) can be placed on the entire Local Author­i­ty Zone in Envoy. This would cause all Learn­er­Per­son­al objects pub­lished for sub­scrib­ing appli­ca­tions at the Local Author­i­ty lev­el to be miss­ing med­ical and per­son­al­ly sen­si­tive information.

Auditing

Bec­ta strong­ly rec­om­mends, but does not require, the estab­lish­ment of the fol­low­ing two roles with­in each school (to para­phrase their document ):

• Senior Infor­ma­tion Risk Own­er (SIRO): This per­son is the pol­i­cy mak­er (typ­i­cal­ly the head teacher) and is famil­iar with the infor­ma­tion risks and pro­ce­dures in place to enforce them. This per­son has appoint­ed Infor­ma­tion Asset Own­ers to enforce these policies.
• Infor­ma­tion Asset Own­ers (IAO): These are the own­ers of the dif­fer­ent types of data: stu­dent per­son­al, med­ical­ly-relat­ed, spe­cial edu­ca­tion, etc.. They need to be able to "ensure that infor­ma­tion han­dling both com­plies with legal require­ments and is used to the full to sup­port the deliv­ery of education".

Hav­ing these respon­si­bil­i­ties is a daunt­ing enough respon­si­bil­i­ty, but not hav­ing the tools need­ed to accom­plish these duties or ver­i­fy that things are "as they should be" is unset­tling at best. So, to address these, Visu­al Soft­ware is adding a set of audit­ing tools to our exist­ing prod­uct set (avail­abil­i­ty expect­ed March 2009). These include :

• Inter­nal audit: this tool allows those respon­si­ble for the pro­tec­tion of data assets to make sure that the school's data is only being dis­trib­uted as it should be from set­tings in Pri­va­cy­Plus. It also gives them access to audits that have been keep­ing track of all traf­fic going through the Zone Inte­gra­tion Server.
• Exter­nal audit: Using the Verac­i­ty Data Integri­ty Man­ag­er, tests can be run to val­i­date that no sen­si­tive data has "leaked out" from any of the providers of information.

Designing for the UK

After spend­ing a short time in the UK, we soon real­ized that a prod­uct line designed for the US mar­ket would only par­tial­ly meet the needs of the UK mar­ket and that the dif­fer­ences between the two were far dif­fer­ent than post code for­mats or the way that con­tacts were stored. We couldn't just take a new data­base schema, update code sets and pre­tend that our prod­ucts met the needs of the UK mar­ket, because they wouldn't. Some of the dif­fer­ences between the two envi­ron­ments run as deep as the dif­fer­ences between the two coun­tries' views on data pri­va­cy. So, in our fee­ble attempt to buck the Amer­i­can tra­di­tion to force our way of doing things on every­one else (you are sup­posed to be at least smil­ing here), we took the path of redesign­ing our prod­ucts to meet the needs and require­ments of the UK mar­ket, from the enhanced data pri­va­cy require­ments to the mul­ti-lev­el region­al zon­ing requirements.

There are three core prod­ucts in a typ­i­cal Visu­al Soft­ware SIF imple­men­ta­tion: ZIS­erv­er (the Zone Inte­gra­tion Serv­er, Envoy (the Mul­ti-Zone man­ag­er) and Verac­i­ty (the Data Qual­i­ty man­ag­er). All three prod­ucts are designed to work on a Microsoft plat­form and can be installed in a small loca­tion using a freely down­load­able SQL Serv­er Express edi­tion or can be spanned across mul­ti­ple large servers form­ing load bal­anced clus­ters and using failover clus­tered data­bas­es and all the things you would expect in a large city implementation.

Each of the Visu­al Soft­ware prod­ucts has com­po­nents of our Pri­va­cy­Plus tech­nol­o­gy, devel­oped to meet or exceed the require­ments of the UK's data pri­va­cy require­ments. We've added it in more than one place to make admin­is­tra­tion more reasonable.

For exam­ple, putting pri­va­cy pro­tec­tion only in the ZIS­erv­er prod­uct would pro­vide the pro­tec­tion required by the DPA and rec­om­mend­ed by Bec­ta, but since there are so many indi­vid­ual end­points to man­age there, it would become very dif­fi­cult to man­age if the ZIS were host­ed. To make the man­age­ment more rea­son­able to man­age and to pro­vide a way for admin­is­tra­tors to pro­vide a fail-safe "blan­ket-fil­ter", we also added it to Envoy. This is added pro­tec­tion, allow­ing for a region­al admin­is­tra­tor to set default poli­cies for all schools in the region as a start­ing place, then allow­ing the ZIS admin­is­tra­tor to "take it from there". Of course, all of these are pol­i­cy set­tings and are at the choice of those who set up the software.

Planning a Successful Implementation

This sec­tion out­lines a rec­om­men­da­tion for the imple­men­ta­tion of the foun­da­tion­al SIF lay­er (the part that gets things going). Imple­men­ta­tion of these parts suc­cess­ful­ly will ensure a much high­er like­li­hood of suc­cess as oth­er appli­ca­tions are added lat­er on.

1. Set up a test envi­ron­ment. If funds are tight, we would sug­gest using vir­tu­al servers and test edi­tions of soft­ware "“ the test edi­tions can usu­al­ly be obtained at a frac­tion of the full cost and the envi­ron­ment for vir­tu­al servers (at least for Microsoft prod­ucts) can be down­loaded free of charge from their web site and can be run from a desk­top or lap­top com­put­er. If you're run­ning in a large orga­ni­za­tion that has host­ed, shared facil­i­ties, you can ask to have test zones cre­at­ed on the shared servers but you should still set up test copies of your applications.
2. Install the Zone Inte­gra­tion Serv­er soft­ware, Envoy and Verac­i­tyCre­ate a "Raw" zone. Since very lit­tle data would pass val­i­da­tion in ini­tial test­ing, we rec­om­mend turn­ing val­i­da­tion off at this point "“ this will be the zone that con­nects the MIS and Envoy. Oth­er agents will nev­er con­nect to this zone because this data will not be pri­va­cy pro­tect­ed nor will its data be passed through any val­i­da­tion testing.
3. Either the MIS or ZIUP becomes provider in this zone "“ this "Raw" zone approach allows con­nec­tion of a feed­ing agent by any SIF-enabled appli­ca­tion "“ whether it is a sup­pli­er-pro­vid­ed SIF agent or ZIUP gen­er­at­ing events on behalf of the MIS. Then import MIS data into Envoy and let it ana­lyze the MIS data.
4. Envoy will sep­a­rate MIS data into "good data" and "bad data", depend­ing if the data would be able to pass cur­rent SIF val­i­da­tion rules. "Good data" will be marked as "OK to pub­lish" and "bad data" will be pre­sent­ed to school users through a Verac­i­ty-like web-based user inter­face. In this way, Envoy is behav­ing like a "qual­i­ty gate­way", only let­ting through objects that meet basic qual­i­ty tests.
5. Have school users clean up data in MIS "“ as school users clean up data in the MIS, the SIF agent for the MIS will auto­mat­i­cal­ly send through SIF "Change" events and the data will move from the "bad data" to the "good data" cat­e­gories and will no longer show up in the school user's user inter­face (no fur­ther user action will be required oth­er than sim­ply cor­rect­ing the mistake).
6. "Clean" zones for oth­er appli­ca­tions to sub­scribe to "“ two or three zones will be cre­at­ed, depend­ing on if Envoy is imple­ment­ed at the Local Author­i­ty or Region­al Broad­band Con­sor­tium lev­el. Once the data for the school is suf­fi­cient­ly clean, the school's Envoy con­nec­tion can now reg­is­ter as a provider in its cor­re­spond­ing "Clean" zones.
7. Before it reg­is­ters as the provider to oth­er appli­ca­tions, the school SIRO reviews the pri­va­cy set­tings for the school in Envoy to make sure that all DPA-defined sen­si­tive infor­ma­tion is prop­er­ly pro­tect­ed. Note that this is all done before any oth­er appli­ca­tions are con­nect­ed, even with­in the same school.
8. After Envoy is con­nect­ed as the provider, the School, LA and RBC copies of Verac­i­ty will be loaded with ini­tial copies of the school's data and ini­tial Verac­i­ty rules for Type 2–5 errors will be run. For LAs and RBC's these will include data pri­va­cy checks to make sure that no DPA-defined sen­si­tive infor­ma­tion has "leaked" into one of the high­er lev­el zones.
9. Con­nect oth­er appli­ca­tions, con­vert data into them and check to make sure all data has made it suc­cess­ful­ly into them.

Com­pare this to a typ­i­cal US installation:

1. Install the ZIS
2. Install MIS agent and sub­scrib­ing agents, most like­ly in a test envi­ron­ment first
3. Con­vert the data into sub­scrib­ing appli­ca­tions and use a lack of infor­ma­tion in the sub­scrib­ing appli­ca­tion to detect prob­lems in the qual­i­ty of the data in the MIS.

Although the sec­ond approach sounds much sim­pler on the sur­face, it has a few problems:

• It leaves the users guess­ing why the sub­scrib­ing appli­ca­tion wasn't able to cre­ate a stu­dent or oth­er object. The cause could have orig­i­nat­ed in one of many SIF objects it sub­scribes to and it will now be the user's respon­si­bil­i­ty to track down which one was the cause of the prob­lem in order to solve the prob­lem. A sig­nif­i­cant amount of time will be spent track­ing down every indi­vid­ual problem.
• It doesn't address data pri­va­cy issues at all, which usu­al­ly isn't as impor­tant in the US as it is in the UK. This mod­el allows all data to go every­where and trusts receiv­ing agents to ignore the infor­ma­tion that is not sup­posed to be deliv­ered to them.
• Some records that look as if they are good are actu­al­ly "almost good" and some of the infor­ma­tion required for them to work effi­cient­ly in the sub­scrib­ing appli­ca­tion is miss­ing. These types of errors are not like­ly to sur­face until the sub­scrib­ing appli­ca­tions are in pro­duc­tion and are being used every day.

This approach is like debug­ging an appli­ca­tion only by look­ing at the out­put "“ you nev­er quite know where a prob­lem orig­i­nat­ed if one exists. You can take guess­es by going back­wards through the process, but in this case every time you move back­wards it might involve a dif­fer­ent sup­pli­er. Con­sid­er the sce­nario from before:

"where each of these pieces was obtained from a dif­fer­ent sup­pli­er. If the error was dis­cov­ered in the VLE, the sup­pli­er of the VLE SIF agent could be the source of the prob­lem. Or, the prob­lem could be in the ZIS, or in the MIS SIF agent, or in the MIS. By using the method­ol­o­gy we sug­gest, we are using meth­ods where­by errors can be caught at each point in the process as soon as they hap­pen, avoid­ing the "snow­ball at the bot­tom of the hill" syndrome.

All in all, we believe that address­ing data qual­i­ty and pri­va­cy issues before any data is dis­trib­uted to oth­er appli­ca­tions auto­mat­i­cal­ly will save time and avoid data secu­ri­ty prob­lems in the long run, thus becom­ing the far low­er cost solu­tion after every­thing is tak­en into consideration.