Real-time SIF-Based State Level Data Collection Tool Set[edit]

Please do not dis­trib­ute a link to this arti­cle to any­one with­out express per­mis­sion from Visu­al Soft­ware, Inc.

Abstract[edit]

Cur­rent­ly, school dis­tricts spend an aver­age of 10% of their admin­is­tra­tive staff costs col­lect­ing, prepar­ing, deliv­er­ing, and man­ag­ing stu­dent and staff infor­ma­tion to meet the require­ments of state and fed­er­al author­i­ties. The most com­mon (if not ubiq­ui­tous) tech­nique used by the states for col­lect­ing this infor­ma­tion is:

• The dis­trict sends col­lect­ed data to a cen­tral­ly loca­tion at the state;
• The state val­i­dates the data and tells the dis­trict where errors exist, giv­ing the dis­trict a cer­tain amount of time to cor­rect these errors;
• The dis­trict makes cor­rec­tions and repeats the sub­mis­sion cycle until all errors have been corrected.

The prob­lem with this method: the errors that caused the rejec­tion by the state are still in the source sys­tems and will there­fore reoc­cur and require cor­rec­tion after every col­lec­tion, ren­der­ing this tech­nique inef­fi­cient and cost­ly. at both the dis­trict and state levels.

Our tool set is based on a mod­el we are imple­ment­ing in the UK that changes the sub­mis­sion cycle from a “batch” process to a “real time” process, allow­ing dis­tricts to cor­rect errors as they occur, so that when the sub­mis­sion is due, the data is released error-free. Fur­ther­more, due to its SIF inter­face, this process occurs automatically.

Commercial Potential[edit]

There is sig­nif­i­cant demand for prod­ucts that col­lect infor­ma­tion from school dis­tricts at the state lev­el as evi­denced by the NCES SLDS Grant pro­gram. As part of our nor­mal busi­ness activ­i­ties in the US, we work with dis­tricts help­ing them con­nect to these sys­tems and sub­mit their data.

In the UK, how­ev­er, we are work­ing with a large cus­tomer solv­ing the same chal­lenge, with the data being sent from the schools to the region­al lev­el in real time while con­tin­u­ous­ly per­form­ing qual­i­ty checks. This process increas­es effi­cien­cy and accu­ra­cy while sig­nif­i­cant­ly reduc­ing costs for both the region and the district.

Significance[edit]

Problem[edit]

All school dis­tricts in the US are required to main­tain detailed and accu­rate stu­dent and staff infor­ma­tion and peri­od­i­cal­ly report a por­tion of this infor­ma­tion to the state, which in turn sum­ma­rizes, ana­lyzes, and reports the infor­ma­tion to the US Depart­ment of Edu­ca­tion. The exact infor­ma­tion and the fre­quen­cy of report­ing dif­fers from state to state, but the require­ment presents a sig­nif­i­cant chal­lenge to dis­tricts espe­cial­ly in light of tight­en­ing bud­gets and over­worked staff.

With the intro­duc­tion of the Schools Inter­op­er­abil­i­ty Frame­work (SIF) stan­dard into this data col­lec­tion process, some of this work­load has been reduced, but we believe that by using SIF with an improved design and by imple­ment­ing some of its new­er fea­tures, much more of the pres­sure can be alle­vi­at­ed and much of the cost can be reduced, both at the state and dis­trict levels.

In this record col­lec­tion process there are two sig­nif­i­cant high cost sources that can be addressed through a prop­er­ly pack­aged solution:

1. cor­rect­ing errors more than once
2. requir­ing dis­tricts to make sig­nif­i­cant local upgrades when they are not necessary

Correcting errors more than once[edit]

The first item is a by-prod­uct of the design of many of the cur­rent SIF-based col­lec­tion sys­tems. These sys­tems sim­ply replace an exist­ing ETL (Extract-Trans­form-Load or CSV upload) process with a SIF con­nec­tor (for exam­ple, see: The WISE State Report Manager[[[1]]]). These solu­tions include a data coor­di­na­tor user role whose job is, on the time report­ing date, to take a sam­ple of the SIS data, sub­mit it to the state, then iden­ti­fy and cor­rect the errors., The prob­lem is that the errors are only cor­rect­ed in the copy of the data by the data coor­di­na­tor and not in the orig­i­nal sys­tem by the source sys­tem operator.

Our tool set removes this role entire­ly by mov­ing the error cor­rec­tion process back to the per­son who made the error, and hav­ing it cor­rect­ed short­ly after it is made. By doing this, the per­son who has the best infor­ma­tion about the object (stu­dent, teacher, course, bus route, etc.) makes the cor­rec­tion. The cor­rect infor­ma­tion is then dis­trib­uted to oth­er depart­ments in the dis­trict through the SIF interface.

"Forklift upgrades"[edit]

Sev­er­al SLDS-relat­ed RFPs we have seen com­pel dis­tricts to per­form SIS or oth­er upgrades to ensure that they all use the same sys­tem, allow­ing the state to sup­port a small num­ber of adapters or SIF agents.

Some oth­er archi­tec­tures require that all dis­tricts use a com­mon Zone Inte­gra­tion Serv­er or non-stan­dard agent behav­ior in order to work properly.

Both of these options can be very expen­sive for the state and the dis­tricts. The best alter­na­tive is to let the dis­tricts use what is best for their needs (typ­i­cal­ly what they have already) and pro­vide the state with some­thing that will work with­in that frame­work, mak­ing very few, or in most cas­es, no demands for new dis­trict-lev­el purchases.

The tool, its use, and the intended outcomes[edit]

The tool set we intend to pack­age dur­ing Phas­es I and II is a real-time data col­lec­tion sys­tem for stu­dent and staff infor­ma­tion based on a sys­tem we are present­ly imple­ment­ing in the Unit­ed King­dom for the East of Eng­land Broad­band Net­work (E2BN) for iden­ti­ty and role man­age­ment and ver­ti­cal reporting.

Real-time Data Collection[edit]

This sys­tem will pro­vide school dis­trict source sys­tem users with con­tin­u­ous feed­back show­ing them how close their oper­a­tional data is to the state-defined qual­i­ty stan­dards, so that when the time comes for a state data col­lec­tion the data from their SIS can be sam­pled and sub­mit­ted with­out any fur­ther adjustments.

Cur­rent­ly imple­ment­ed state data col­lec­tion sys­tems, even those imple­ment­ed using the Schools Inter­op­er­abil­i­ty Frame­work (SIF), do not pro­vide such a con­tin­u­ous feed­back cycle. This results in:

• Errors not being cor­rect­ed in the source systems—the peo­ple who cor­rect the errors in the sub­mis­sion sam­ple do not have time or per­mis­sion to edit the SIS records to cor­rect the errors there. Addi­tion­al­ly, so much time would have elapsed between the sub­mis­sion cut­off date (when the sam­ple was tak­en) and when the cor­rec­tions would be made, nobody would be sure if the change is still valid.

• Errors being cor­rect­ed in the sam­ple tak­en on the sub­mis­sion date and not in the source sys­tem, because the error feed­back was only sent back for the sam­ple and not for the live data.

• Dis­tricts wast­ing large amounts of effort and mon­ey need­less­ly re-cor­rect­ing the same errors every time a new sub­mis­sion is due.

Working with In-place District Software[edit]

The solu­tion we are propos­ing is designed to work with what­ev­er SIS soft­ware is cur­rent­ly in place with very few (if any) exceptions.

After work­ing with dozens of ven­dors over many years, we have found that a rea­son­able expec­ta­tion is that they can all cre­ate Com­ma Sep­a­rat­ed Val­ue (CSV) export files for school, stu­dent and teacher infor­ma­tion on an auto­mat­ed basis and place that infor­ma­tion in a des­ig­nat­ed direc­to­ry (they need to do this to meet cur­rent state report­ing requirements).

Our tool set will include options for dis­tricts that have:

• Com­plete SIF imple­men­ta­tions – the state data col­lec­tor­will become an active mem­ber in the district’s zone.

• A SIS that has a SIF agent in a dis­trict has no SIF imple­men­ta­tion' – the state will host a SIF zone for that dis­trict and the SIS will con­nect to it.

• A SIS that can only pro­duce extract files – the state will pro­vide a “CSV SIF agent” (one of our com­po­nents) that will pro­vide SIF agent ser­vices for those types of applications

District “Buy-In”[edit]

When one orga­ni­za­tion requires anoth­er to install or use some new soft­ware it is bet­ter if the new user feels that it is worth the effort to learn a new sys­tem and that the new­er sys­tem is an improve­ment over the old.

Giv­ing reg­u­lar feed­back to source appli­ca­tion users about errors that need to be cor­rect­ed for state report­ing would be a marked improve­ment. Under­stand­ing that this toolset would also reduce or elim­i­nate the bar­rage of phone calls and emails about errors found that usu­al­ly accom­pa­ny a sub­mit­tal should sub­stan­tial­ly increase “buy-in” poten­tial. We have seen first-hand reac­tions of many users when they first saw this capa­bil­i­ty and over­whelm­ing­ly their reac­tion has been very positive.

Privacy Concerns[edit]

One of the issues some have with a real-time data col­lec­tion mod­el is a con­cern for stu­dent pri­va­cy. This was one of the ear­ly-stage obsta­cles we faced while first imple­ment­ing our project in the UK. The con­cern is that the state will gain access to pri­vate infor­ma­tion while in the process of gath­er­ing real-time infor­ma­tion and mon­i­tor­ing the qual­i­ty of the data. SIF, in its orig­i­nal design allowed a sub­scriber (such as a state) to receive all infor­ma­tion in an object if it received any of the infor­ma­tion in that object, includ­ing per­son­al infor­ma­tion which is pro­tect­ed by law.

To address this con­cern and to make real-time col­lec­tion pos­si­ble, Visu­al Soft­ware intro­duced the con­cept of “ele­ment-lev­el fil­ter­ing” to SIF in 2008 and led the effort to have it adopt­ed as part of the stan­dard in 2009. Since then, ele­ment-lev­el fil­ter­ing has been adopt­ed by all of the major SIF ZIS providers and is uni­ver­sal­ly avail­able today.

Ele­ment-lev­el fil­ter­ing allows the state to become an active mem­ber in a district’s zone, but only able to receive infor­ma­tion for the par­tic­u­lar data ele­ments that it needs to see / is allowed to see by law.

Shared vs. Private Facilities — Scalability[edit]

Through the work we have done with cus­tomers such as Lon­don (2,700 schools), the East of Eng­land (3,200 schools), the East Mid­lands (3,000 + schools) in the UK and Vic­to­ria, AU (1,600 schools, all in pro­duc­tion), we have found that many of the facil­i­ties that are com­mon­ly installed indi­vid­u­al­ly at school dis­tricts in the US can eas­i­ly be shared at state-wide or region­al host­ing cen­ters at a much low­er aver­age cost, while still pro­vid­ing the same or bet­ter lev­el of service.

Description of the Tool Set — Architecture Overview[edit]

The solu­tion we are propos­ing is com­posed of a com­bi­na­tion of con­fig­urable prod­ucts cur­rent­ly made and sold by Visu­al Soft­ware, but at present not being used for state ver­ti­cal report­ing in the Unit­ed States. These prod­ucts include :

• ZIS­erv­er™: enter­prise lev­el SIF Zone Inte­gra­tion Serv­er, capa­ble of host­ing thou­sands of schools in a sin­gle imple­men­ta­tion (will be used as the cen­tral mes­sage han­dling hub);
• Verac­i­ty™: SIF-enabled data col­lec­tion and data qual­i­ty man­age­ment tool (will be used for col­lect­ing and qual­i­ty check­ing the data and for pro­vid­ing a user inter­face to the districts);
• Ipse­ity™: SIF-enabled iden­ti­ty and role man­age­ment sys­tem (will be used for man­ag­ing accounts for all dis­trict staff mem­bers that need access to any of the host­ed prod­ucts, so that sin­gle sign-on access can be pro­vid­ed and accounts can be main­tained through the sub­mit­ted data);
• Envoy™: SIF-enabled Vir­tu­al Zone Man­age­ment sys­tem (will be used for dis­tricts with no SIF imple­men­ta­tion, so that school infor­ma­tion can be con­sol­i­dat­ed at the dis­trict lev­el in real time before it is trans­mit­ted to the state);
• Mim­ic™: SIF agent for appli­ca­tions that only have an abil­i­ty to pro­duce CSV files (will be used in those dis­tricts where the imple­ment­ed SIS does not have a SIF agent available).

Each of these prod­ucts is already devel­oped, test­ed, and in pro­duc­tion; some of them, how­ev­er, are only offered for non-US ver­sions of the SIF spec­i­fi­ca­tion. The dia­gram to the right illus­trates how the prod­ucts would work togeth­er in a state deployment.

This dia­gram illus­trates a part of a deploy­ment with three dis­tricts (on the left) and a state col­lec­tion cen­ter (on the right). The soft­ware that would be part of our solu­tion has a high­light­ed back­ground and the appli­ca­tions that would be host­ed at the state are marked as "Host­ed".

School District Type A[edit]

The first school dis­trict (the one pic­tured with "SIS A") has a SIF imple­men­ta­tion and a local­ly installed SIF Zone Inte­gra­tion Serv­er (ZIS). Present­ly this cor­re­sponds to more than 10 per­cent of the school dis­tricts in the US with very rapid growth. In this sce­nario, the data col­lec­tor / qual­i­ty mon­i­tor would become a sub­scriber in the school's SIF zone.

Verac­i­ty would sub­scribe to the SIF objects that have infor­ma­tion required by the state and the dis­trict would be instruct­ed how to imple­ment its ele­ment-lev­el fil­ters for the con­nec­tion (all the major ZIS ven­dors have already imple­ment­ed ele­ment-lev­el fil­ters). Verac­i­ty could have tests to make sure that the ele­ment lev­el fil­ters were prop­er­ly set up.

School District Type B[edit]

The sec­ond school dis­trict (the one pic­tured with "SIS B") does not have a SIF imple­men­ta­tion but has a Stu­dent Infor­ma­tion Sys­tem that has a SIF agent avail­able for it (present­ly this describes about 50–60% of the school dis­tricts in the coun­try). In this sce­nario, the state would host a ZIS, cre­ate a zone for that school dis­trict and the district's SIF agent would reg­is­ter in that zone as a publisher.

In this sce­nario, the des­ig­nat­ed site admin­is­tra­tor would be allowed to admin­is­ter not only the Verac­i­ty access, but also the ZIS­erv­er admin­is­tra­tion access for the district's zone as if it was a Zone Inte­gra­tion Serv­er locat­ed at their dis­trict office. (ZIS­erv­er is also pro­grammed with sin­gle sign-on fea­tures so that indi­vid­ual zones can be admin­is­tered remotely).

School District Type C[edit]

The third school dis­trict (the one rep­re­sent­ed with "SIS C") rep­re­sents those that have a cus­tom-made or Stu­dent Infor­ma­tion Sys­tems, but that do not have or nev­er will have a SIF agent. We have found that such sys­tems can already cre­ate CSV-file export files because of pre­vi­ous state report­ing requirements.

Although, the stu­dent infor­ma­tion is com­bined at the dis­trict lev­el, in a few of these dis­tricts, most of these places will also have the added com­pli­ca­tion of hav­ing sep­a­rate stu­dent data­bas­es at each school. This is why we rec­om­mend­ed Envoy for these school dis­trict. Envoy is a SIF-enabled prod­uct that man­ages the con­sol­i­da­tion of stu­dent, staff and par­ent records for mul­ti­ple schools in a larg­er organization.

Note: Envoy is one of the appli­ca­tions that is present­ly only avail­able in non-US edi­tions and would need to be con­vert­ed for the US as part of this project.

Intended Audiences[edit]

There are two audi­ences for our tool-set: one at the dis­trict and the SEA.

District Audience[edit]

Most of the peo­ple who use this tool set at the dis­trict lev­el would be the users of source sys­tems that gen­er­ate the data for state reports. Addi­tion­al­ly, users would include IT staff who con­trol role assign­ments and data man­age­ment staff who would over­see the admin­is­tra­tion of the data col­lec­tion process.

When com­pared to the exist­ing meth­ods of data col­lec­tion, more peo­ple will come in con­tact with the ver­ti­cal report­ing soft­ware, but each of them will only have a small frac­tion of the work to perform.

State or Regional Audience[edit]

At the loca­tion where report­ing data is gath­ered from the dis­tricts, the users of this tool set would be those who man­age the data and feed it into the region's or state's data warehouse.

State-lev­el users will par­tic­i­pate in devel­op­ing busi­ness rules for check­ing data. In pre­vi­ous data col­lec­tion projects we have done using this, we have found that once users under­stand the capa­bil­i­ties of the soft­ware, the num­ber of checks tends to grow with each new data sub­mis­sion cycle.

The District Daily User Experience[edit]

(note: you may click on any image to see infor­ma­tion about the image, then again to see the image full-screen)

Click to see full-size image

Application System Users[edit]

From the dis­trict user's per­spec­tive, he or she would have dai­ly access to a por­tal site (host­ed by the state) that shows how far away his or her district's data is from being clean enough to sub­mit direct­ly to the state for the next sub­mis­sion. The exam­ple to the right is tak­en from our UK edi­tion of Verac­i­ty with some basic data rules we set up that cor­re­spond to their stu­dent Every Child Remem­bered cen­sus (the equiv­a­lent of a No Child Left Behinddata col­lec­tion).

The data is auto­mat­i­cal­ly fed from the Stu­dent Infor­ma­tion Sys­tem and oth­er con­nect­ed sys­tems (such as HR or Café sys­tems) through the SIF con­nec­tors to the SIF real-time data col­lec­tor. As the data arrives, Verac­i­ty applies data qual­i­ty rules against the incom­ing data and main­tains a sep­a­rate data­base of cur­rent exceptions.

When the user logs into the dis­trict por­tal, he or she sees any out­stand­ing errors and drill down to deep­er lev­els of detail to see enough infor­ma­tion to cor­rect the error in the source sys­tem. When the error is cor­rect­ed, the cor­rec­tion will flow the SIF con­nec­tion through to Verac­i­ty (and all oth­er con­nect­ed appli­ca­tions). The error will then drop off the error list a few sec­onds later.

Districts without SIF-enabled Student Information Systems[edit]

For places where the Stu­dent Infor­ma­tion Sys­tems do not have SIF inter­faces we devel­oped a prod­uct called Mim­ic that works with the inter­face that most of these sys­tems do have — the abil­i­ty to cre­ate CSV on an auto­mat­ed basis. Mim­ic can be set up in a few hours using a wiz­ard-like inter­face and runs on a min­i­mal­ly-con­fig­ured PC. It assumes that the SIS will gen­er­ate on a peri­od­ic basis:

• a list of new and updat­ed records only (only adds and changes will be pub­lished), or

• a com­plete dump of the object (adds, changes and deletes will be published).

At a des­ig­nat­ed inter­val, Mim­ic will reg­u­lar­ly inspect the CSV file direc­to­ry; when it sees a new file; it will com­pare it against the last one it saw there, then pro­duce SIF events cor­re­spond­ing to the dif­fer­ences between the two. It will also respond to requests from the SIF zone.

Single sign-on and Data Scoping[edit]

When a dis­trict user logs in, all Visu­al Soft­ware tools will use the infor­ma­tion received about an employ­ee through the SIF inter­face to auto­mat­i­cal­ly scope the data he or she is allowed to access. All users are auto­mat­i­cal­ly scoped based on dis­trict; as part of Phase II, we will be adding anoth­er lay­er of scop­ing which will lim­it a user’s access by func­tion as well.

User Alerts[edit]

Our cur­rent Verac­i­ty prod­uct has the func­tion­al­i­ty to alert users to the pres­ence of out­stand­ing errors. As part of Phase II, we are con­sid­er­ing the addi­tion of an RSS feed as well (to be includ­ed with the user scop­ing fea­ture). This will allow school- or dis­trict-based staff to set up an alert feed so that they will be con­tin­u­al­ly remind­ed when­ev­er there are any unre­solved issues with stu­dent data. He or she will need to login into the por­tal site to find out the par­tic­u­lar stu­dent details, but the RSS feed will keep him or her informed as to the nature and quan­ti­ty of the errors.

The District Collection Experience[edit]

On cer­tain cut­off dates, data sam­ples are tak­en from the dis­trict and are pre­pared to be giv­en to the state (or region­al col­lec­tion author­i­ty) (the SEA). The SEA expects the district's data to be "as of" that date, so even if the dis­trict is using SIF as a col­lec­tion method, some sam­ple will need to be tak­en on that date. Typ­i­cal­ly, the SEA will give the dis­trict a peri­od of time to resolve any issues with the data before the data is final­ly due.

Our archi­tec­ture changes the cycle of activ­i­ty as follows:

When the report­ing cut­off date is reached, the dis­trict user will see a new tab appear­ing on his or her screen. This is because at mid­night that evening a snap­shot of the data that was sit­ting in the district's Verac­i­ty data­base was tak­en and moved to a "report­ing col­lec­tion area". In this exam­ple, the user inter­face in the screen shot illus­trates a user's view (in the UK) just after a Decem­ber 2010 snapshot.

If you look close­ly at the screen shot on the next page, you will see that the "Dec 2010" tab has been select­ed and that the "Sum­ma­ry — Dis­tri­b­u­tion" option has been select­ed from the menu at the left of the page. Doing this caus­es Verac­i­ty to show the dis­tri­b­u­tion of errors that still remain in the dataset as of the data col­lec­tion date (in a live dis­trict, this should be much clos­er to zero — this exam­ple uses test data for illus­tra­tive pur­pos­es). If the "Sum­ma­ry — Dis­tri­b­u­tion" option were to have been with any of the oth­er tabs, this graph would have shown a dis­tri­b­u­tion of errors for that time peri­od or the cur­rent (live) data.

Click thumb­nail to show larg­er screen image

If any errors remain at this point, the user can edit the sam­ple that was tak­en and those changes will be made inde­pen­dent­ly of the source sys­tems. This is not opti­mal, but it is required because the "as of" date of the sam­ple must be main­tained. The user may get to the errors either by drilling down through the list of errors or by using Veracity's "search" inter­face to enter any of the char­ac­ter­is­tics of the object (learn­er last name, for exam­ple) to find the record, then edit its con­tents through a web form inter­face. All inter­ac­tions are record­ed so that the state and dis­trict have a com­plete record of all changes that were made to records after the snap­shot was taken.

The fol­low­ing is an exam­ple of the search/edit interface:

Click thumb­nail to show larg­er screen image

The State (SEA) User Experience[edit]

At the SEA, the web inter­face is sim­i­lar to that which the user sees, except that the data now spans mul­ti­ple dis­tricts and the data qual­i­ty rules also span dis­tricts. The pre­cise state user expe­ri­ence will vary, depend­ing on state law and if the state will be allowed to see the dis­tricts’ live data before the report­ing cut-off date.

Privacy Issues and Data Filtering[edit]

Log­i­cal­ly (and phys­i­cal­ly), the SEA main­tains a sep­a­rate instance of Verac­i­ty con­nect­ed to a statewide-SEA SIF zone (phys­i­cal­ly host­ed on the same ZIS serv­er but log­i­cal­ly sep­a­rate). Each school dis­trict host­ed Verac­i­ty instance relay events they receive into this zone, but the objects are fil­tered so that only the fields that the SEA has per­mis­sion to see are relayed.

The state piece of the archi­tec­ture would look like the pic­ture to the right.

Each dis­trict is rep­re­sent­ed by a zone (either at their dis­trict on their ZIS or on the shared ZIS at the SEA). The "Dis­trict Verac­i­ty" copy con­tains the com­plete details of their stu­dent infor­ma­tion and only school dis­trict per­son­nel have access to view this infor­ma­tion. When mes­sages are sent to the "Dis­trict Verac­i­ty" instances, fil­tered ver­sions of them are relayed to the "Com­bined Verac­i­ty" copy.

Business Rules that Span District Boundaries[edit]

This SEA Verac­i­ty copy has a sep­a­rate set of busi­ness rules that can check for things such as:

• A stu­dent that has more than one school des­ig­nat­ed as his or her "main school".
• A stu­dent is marked as "grad­u­at­ed from high school" with­out required tests being tak­en (this infor­ma­tion may come from records in dif­fer­ent districts).

When Data is Available in Real-Time[edit]

If state law allows fil­tered live data to be made avail­able to the state, sev­er­al new pos­si­bil­i­ties could also arise:

• State Stu­dent Iden­ti­fiers could be assigned when they are need­ed with­out human intervention.
• Stu­dent mul­ti­ple enroll­ments could be checked in real time, alert­ing dis­trict staff (in both schools) imme­di­ate­ly when chil­dren move out of the dis­trict with­out notification.

Auditing Activities[edit]

Click thumb­nail to show larg­er screen image

All activ­i­ty, either at the LEA or SEA lev­el is logged by default. This includes all XML mes­sage traf­fic as well as all human inter­ac­tions. We rec­og­nize that this gen­er­ates a very large vol­ume of infor­ma­tion and that some of these audits are redun­dant. In the UK, each of these lev­els of audit­ing is required by law; in the US they are not, so we make the lev­el of audit­ing a con­fig­u­ra­tion option.

Support Materials[edit]

Over the past sev­er­al years, Visu­al Soft­ware has devel­oped and refined sev­er­al sys­tems for pro­vid­ing high qual­i­ty online sup­port for our cus­tomers around the world, includ­ing many with whom we share few busi­ness hours in com­mon (the Unit­ed King­dom, Egypt and Aus­tralia). These include :

• An online help desk sys­tem where they can report prob­lems and track the progress of any open tickets;
• A wiki site with hun­dreds of detailed tuto­ri­als illus­trat­ing how to do the things our cus­tomers need to do on a dai­ly basis;
• A soft­ware down­load por­tal where cus­tomers can go to get the lat­est releas­es and detailed notes about what has been changed from release to release;
• A doc­u­men­ta­tion site that helps cus­tomers in plan­ning imple­men­ta­tions and gives them insight into new prod­ucts we're developing.

Cre­at­ing these facil­i­ties to be as good as they are (and as attrac­tive as they are to our users) has become a key part of Visu­al Software’s “scal­a­bil­i­ty” strat­e­gy. The more that our cus­tomers are drawn to this site when they need help (and find what they need there), the more of them we can sup­port with a sin­gle inter­nal staff member.

Intended Outcomes[edit]

Dis­tricts and SEAs that use this tool set should expect to see a marked improve­ment in data accu­ra­cy with some improve­ments in effi­cien­cy in their first report­ing cycle. By the sec­ond report­ing cycle, sig­nif­i­cant improve­ments in effi­cien­cy should be noticed.
Dur­ing the first report­ing cycle, when an error occurs, it will be brought to the atten­tion of the per­son oper­at­ing the source appli­ca­tion sys­tem short­ly after the orig­i­nal data was entered (not weeks lat­er). The cor­rect infor­ma­tion will be eas­i­er to locate and the error will be eas­i­er to cor­rect. No call will need to be made (as when a data coor­di­na­tor calls the source appli­ca­tion oper­a­tor) to find out the cor­rect infor­ma­tion and the cor­rec­tion will be made using the SIS inter­face and not by edit­ing a CSV file in a text editor.

Dur­ing the sec­ond report­ing cycle, because these recur­ring errors were per­ma­nent­ly fixed in the source sys­tem, only new errors that occur dur­ing the report­ing cycle will need to be corrected.

Types of Errors[edit]

For­mat errors: These are the tough­est type of error to han­dle when work­ing with CSV files — they typ­i­cal­ly affect an entire col­umn of data and cause hun­dreds or thou­sands of errors in the dataset. They are usu­al­ly the result of an error in the soft­ware that gen­er­ates the export file and not of data entry, although some­times it is hard to tell the dif­fer­ence. With a CSV inter­face, typ­i­cal­ly the entire batch is dis­card­ed if any errors of this type occur and it typ­i­cal­ly requires sub­stan­tial inves­ti­ga­tion by some­one to find out what the prob­lem is. With SIF/XML, the exact type of error is usu­al­ly eas­i­ly identified.

Val­i­da­tion errors: It is best for SIS to include val­i­da­tion checks for as many of these fields as pos­si­ble direct­ly in their appli­ca­tions so that the errors can be caught at the time of entry, but many do not for one rea­son or anoth­er. Some­times, the data col­lect­ed by an SEA is unusu­al and not nor­mal­ly need­ed by a SIS to do its nor­mal job (for exam­ple, the results of a blood test). In cas­es like this, the SIS stores these val­ues in "user defined fields" that under­go lit­tle or no val­i­da­tion at the point of entry. As a result, when it is deliv­ered to the SEA as part of a sub­mis­sion, that field's con­tents may not be very consistent.

Anoth­er exam­ple of a val­i­da­tion error that is dif­fi­cult to catch is one where the data in one field is deter­mined to be cor­rect or incor­rect based on a val­ue some­where else, per­haps in the same record but per­haps in anoth­er record for the same stu­dent. For exam­ple, see­ing a stu­dent enrolled in "Eng­lish as a Sec­ond Lan­guage" might not make sense if the same student's Pri­ma­ry Lan­guage is record­ed as "Eng­lish". The demo­graph­ics record for that stu­dent was record­ed at one time, but the enroll­ment in that course may have come at a much lat­er time. Also, these might come from two dif­fer­ent applications.

Match­ing and oth­er cross-dis­trict errors: These errors span out­side the bound­ary of the local, live data. They could involve the com­par­i­son of a student's record with those of stu­dents in oth­er dis­tricts (dupli­cate enroll­ments) or that student's pre­vi­ous records from pre­vi­ous col­lec­tions (is this record con­sis­tent with pre­vi­ous submissions).

NOTE: The data that is used in these com­par­isons will only be able to be made using the small sub­set of fields that are in the "fil­tered" set that relayed to the state copy of Verac­i­ty.

Theoretical and empirical Support[edit]

Data qual­i­ty and the process from raw school data to infor­ma­tion, three key play­ers can be iden­ti­fied – Col­lec­tors, Ana­lyz­ers and Users. Under­stand­ably, data col­lect­ed in a con­sis­tent and time­ly man­ner is the most opti­mal and will pro­vide the great­est oppor­tu­ni­ty for