State Longitudinal Data System Development: Custom Build or Off the Shelf
Buy the components and build only the parts that carry your statute. Adopt Ed-Fi for transport and model, adopt Common Education Data Standards definitions, and evaluate eScholar for identity resolution rather than writing matching from scratch.
On this page
Buy the components and build only the parts that carry your statute. Adopt Ed-Fi for transport and model, adopt Common Education Data Standards definitions, and evaluate eScholar for identity resolution rather than writing matching from scratch. Build accountability rule processing, reproducible publication, suppression and the district feedback loop, because those change whenever your legislature acts.
What the off the shelf products actually do well
Say the buy case first, because it is larger than most procurement documents admit. A single district, or a small state buying a warehouse rather than standing up a system of record, should adopt a commercial product and stop there. Most of what an education agency wants from a dashboard already exists, and building it is a slow way to arrive at a chart.
The components are genuinely good. The Ed-Fi data standard, with its operational data store and application programming interface, gives you a model that district student information systems such as PowerSchool and Infinite Campus already support, and adopting it is worth far more than a bespoke schema. The Common Education Data Standards give you element definitions that align with federal reporting instead of a dictionary you maintain alone. eScholar's identity resolution has run in several states and is credible. Public Consulting Group and firms like it deliver real outcomes on the services side. Commercial warehouses and reporting tools cover storage and visualisation without argument.
Buy all of that. The mistake is assuming the remainder is a thin layer on top. It is not, and it is the part that gets you into difficulty.
Where they stop: the number you published three years ago
A superintendent of a district with forty thousand students disputes a four year graduation rate on your public dashboard. He says eleven students in the denominator transferred out of state and his staff hold the documentation. Your team now has to reproduce a cohort computation from three years ago, using submissions that have been resubmitted four times since, under a business rule the legislature amended in the intervening period, with an identity resolution process that merged two records in between.
If you cannot reproduce the published figure exactly, you withdraw it. Withdrawing a published accountability number is the kind of event that changes who runs your office. That is the workflow no product models, because reproducibility is not a feature of a warehouse. It is a discipline that has to be designed in from the first submission.
Two more parts sit outside any vendor's model. Cohort membership rules, what counts as a documented transfer, which assessments substitute for which, and the minimum group size for reporting are written in your state's law and your accountability plan under the Every Student Succeeds Act, not in a configuration screen. And linkage to wage outcomes is a legal design rather than an integration: matching to unemployment insurance wage records, to public postsecondary enrolment and completion, and sometimes to early childhood data is governed by data sharing agreements, by exceptions under the Family Educational Rights and Privacy Act such as audit and evaluation, and by state statute that may forbid the exact join a legislator has just requested in a hearing. Build the join first and ask counsel afterwards and you end up with a system you are not allowed to run.
The arithmetic: districts and records against a phased build
The pricing question here is not per seat, because your users are a small central team. It is per record and per district. Commercial platforms in this space price against enrolment counts or against the number of local education agencies you onboard, with implementation charged separately and each new collection treated as configuration work.
Say your enrolment is 700,000 students across 180 districts, and your platform arrangement plus the annual services retainer runs $600,000 a year. Over five years that is $3M with no source code at the end of it. Then price the labour it does not remove: staff chasing validation errors by email, analysts rebuilding cohort computations by hand each publication cycle, and a suppression review done manually across dozens of report types. In most agencies that is three to five full time positions doing work that exists only because the loop is broken.
The crossover sits near 150 local education agencies, or roughly 400,000 student records under management, whichever you cross first, and it arrives immediately if you have a statutory duty to publish accountability results. Below that, buy and configure. Above it, the recurring cost scales with enrolment and district count while a build does not, and every rule change becomes a change order rather than an afternoon.
What a custom build actually costs
From Digital Heroes delivery experience with public sector data platforms, a first production component runs $200,000 to $450,000 and takes 5 to 9 months. That should almost always be submission intake with a district facing validation workspace plus the identity spine, not a dashboard. A full platform covering education through workforce linkage, accountability rule processing with reproducible publication, public reporting and a researcher request and access workflow runs $700,000 to $2M phased over 18 to 36 months, usually against federal grant funding.
Data migration runs 10 to 25 percent of the build and sits at the very top of that band here, because historical migration is where the identity work concentrates. Twenty years of legacy identifiers have to be resolved rather than imported, and resolution means probabilistic matching, a human review queue for the ambiguous band, and merges recorded as reversible events with full history. A merge that combines two children into one record is worse than a duplicate, and you will find some of those in the archive.
Year two and after runs 15 to 20 percent of build cost annually. Legislatures meet, federal plans get revised, collections change, and accessibility standards for public reporting tighten. What pushes the initial figure up: whether you can mandate a submission standard, since a state that can require Ed-Fi has a materially cheaper project than one accepting whatever arrives; the number of partner agencies, because every interagency agreement is a legal negotiation measured in quarters while the engineering waits; and public dashboards, which look cheap and are not once conformance and defensible suppression are counted.
The four situations where building wins
- Regulatory fit. Your accountability rules are statute. Cohort definitions, assessment substitution, minimum group size and complementary suppression all have to be versioned, reviewable and reproducible, with every published artefact recording which rule version produced it. Access control should derive from the governing data sharing agreement itself, so a query combining two datasets whose agreements forbid combination is refused by the platform with the agreement named.
- Scale economics. Past 150 districts or 400,000 records, the recurring cost scales with the population you serve while the cost to build does not, and each amendment becomes a procurement rather than a change.
- A workflow that is your competitive advantage. For an agency the equivalent is credibility. Being able to recompute any published number from a manifest recording input dataset versions, rule version, identity graph version and code release, and compare it byte for byte to the original, is what lets you answer a superintendent in a day rather than withdrawing a figure.
- Integration sprawl across three or more systems. District student information systems, your assessment vendor, the workforce agency holding wage records, postsecondary systems and federal reporting are separate worlds with separate governance. Nobody sells the joins, and the joins are the point of the whole programme.
For most state agencies, two or more of these are already true. That is why this category is built rather than bought, and why the buy decision applies to the components rather than the system.
How to decide in a week
Run one test and it will settle the argument. Choose a graduation rate or an assessment proficiency figure your agency published three years ago for a named district. Give your data team five working days to recompute it and produce an exact match, along with a written account of the inputs, the rule version and the code that produced the original.
Then look at what the exercise required. Whether the input data as it stood could be retrieved, or only the current corrected version. Whether the rule as it stood was written down anywhere other than in a stored procedure. Whether any identity merges since publication were reversible and identifiable. If the number reproduces exactly and you can show your work, your current arrangement is sound and you should extend it rather than replace it. If it does not reproduce, you have found the reason to build and the sequence to build in, which is intake and identity first and dashboards last.
Then buy a paid discovery phase before any development. Four to six weeks at a fixed fee for a programme this size, and the deliverable is a signed product requirements document: the identity resolution approach with threshold governance, the publication manifest design, the rule versioning model, the governance metadata driving access control, accessibility targets for public reporting, acceptance criteria and a fixed price. Digital Heroes writes that before any code and the agency owns it whichever firm builds. We are wrong for you if you want a supplier to staff your data office as a managed service, or one with an office in your state capital. We work through India LLP, US LLC and UK LTD entities so intellectual property assigns under your own law, and we never claim a local office. More than fifty specialists, over 2,000 projects, a named team you meet before signing, and public records on Clutch, Trustpilot, Fiverr Vetted Pro and D-U-N-S.
Book a 30-minute call with Digital Heroes and get a written plan and a fixed quote within 48 hours.
The evidence behind this guide
Independent findings on why this investment pays off. Every link goes to the primary source.
- Deloitte reports that modern ERP implementations aim to deliver reduced manual effort, greater transparency, a single source of truth, and increased productivity, but many organizations do not capture the full expected benefits (a significantly lower ROI) without disciplined strategy, change management, and data readiness. Source: Deloitte (2024) →
- McKinsey found that tech debt can amount to 20-40% of the value of a company's entire technology estate before depreciation, and CIOs report that 10-20% of the budget for new products is diverted to resolving tech-debt issues. Source: McKinsey & Company (2020) →
- Digital Champions expect to achieve about 16% in cost savings and around 15% in revenue gains from digital operations over five years; the study surveyed 1,155 manufacturing executives across 26 countries. Source: PwC / Strategy& (2018) →
- Across ten outpatient clinics the mean no-show rate was 18.8%, and the marginal cost of no-shows reached $14.58 million per year for those clinics, at roughly $196 per missed appointment (2008 figures). Source: BMC Health Services Research / PubMed Central (Kheirkhah et al.) (2015) →
Frequently asked questions
Who owns the source code when a vendor builds a state data system?
The agency should own the source code, the infrastructure accounts, the documentation and the unrestricted right to hire another firm, with knowledge transfer written as an acceptance criterion rather than a good intention. At Digital Heroes the agency owns everything from the first commit. This matters more in government than anywhere else, because these systems outlive administrations and a rebuild driven by lock in lands on a future commissioner.
How long does an interagency data sharing agreement take to negotiate?
Quarters rather than weeks, and the engineering waits on it. Every partner agency has its own counsel, its own statutory constraints and its own appetite for risk, and the workforce agency is usually the slowest because wage records carry the tightest rules. Start the legal work at the same time as discovery, not when the linkage sprint appears on a plan.
What happens if an identity merge from three years ago turns out to be wrong?
In a well designed system the merge was recorded as a reversible event with full history, every downstream result stored the identity graph version it used, and you can therefore list exactly which published numbers were affected. In a system built on destructive updates you are guessing. Ask any prospective developer this question early, because the answer reveals how they think about time.
Can we phase this against federal grant funding?
Yes, and most agencies do. The practical constraint is that grant periods rarely align with the sequence the work actually needs, which is intake and identity first and public reporting last. Write the phasing so each funded component is independently useful and independently defensible, rather than promising a dashboard in a period where the data underneath it will not yet be trustworthy.
How do we handle districts that submit late or badly?
By moving the feedback loop closer to the person who has to fix it. Validation should run at submission and return errors in the district's own language, meaning student and school names rather than surrogate keys, in a workspace showing every open issue, who owns it and how the same district looked last collection. Certification then becomes an explicit recorded act by a named district official.
What is the difference between a data warehouse and a system of record?
A warehouse stores and reports on data someone else certified. A system of record accepts submissions, resolves identity, applies rules and produces results that carry legal consequences, which means it has to reproduce any published figure years later from versioned inputs, rules and code. Agencies that procure a warehouse and expect system of record behaviour discover the gap during their first challenge.
Should we use a services firm rather than building in house?
Either can work, and the risk with a pure services model is not competence, it is that the knowledge leaves when the contract ends and the next amendment becomes another procurement. If you contract this way, make documentation, source code delivery and knowledge transfer contractual deliverables with acceptance criteria attached, and budget for staff who can read the code after handover.
Do public dashboards really need their own budget line?
Yes, and underestimating them is common. Accessibility conformance is a legal expectation rather than a preference, and retrofitting it is far more expensive than building to it. Defensible suppression is the other half: complementary suppression has to be applied as a versioned service at publication so a reader cannot subtract one table from another to recover a hidden cell.
Can we link to wage records without holding wage data ourselves?
Frequently that is the only lawful design. Where statute prevents the education agency from holding wage records, the pattern is a token or hash based match executed inside the workforce agency environment returning aggregates only. Where a linked research file is permitted, it lives in a separately secured environment with a defined retention clock. Your counsel decides which, and the engineering follows.
Who is Digital Heroes wrong for?
Agencies wanting a supplier to staff their data office as an ongoing managed service, since we build systems and hand them over with the knowledge transfer written into acceptance. Also anyone whose procurement requires an in state office. We work through India LLP, US LLC and UK LTD entities so intellectual property assigns under your own law, and we never claim a local office anywhere.
How long does it take to build a custom BI dashboard?
A working first version usually ships in 4 to 8 weeks, and a full production build with multiple integrations and permissions takes 3 to 6 months. In Digital Heroes delivery experience, schedules slip on data access, meaning credentials, API approvals, and cleanup of source data, far more often than on the dashboard screens themselves. Lining up access to every data source before kickoff routinely saves 2 to 3 weeks.
We run everything on spreadsheets and Airtable. How do we know it's time for custom software?
The reliable signals are re-typing the same data into multiple tools, one employee acting as human middleware between systems, and errors appearing in handoffs between teams. Hard limits force the issue too: Airtable's Team plan caps at 50,000 records per base, and Business costs $45 per seat per month, so a 20-person team pays about $10,800 a year for a tool it has already outgrown. When workarounds consume more hours than the tools save, the spreadsheet era is over.
How many people does it take to build a custom BI dashboard?
A typical build runs with 3 or 4 people: a data engineer for pipelines and modeling, a full-stack developer for the application and charts, a part-time designer, and a project lead. One strong freelancer can handle a single-source internal dashboard, but in our experience solo builds stall once multiple integrations, permissions, and customer access are added. Team size matters less than having one person explicitly own the data model.
How do I vet a software development agency before signing a contract?
Ask to speak with two past clients whose projects resemble yours in size and industry, and ask exactly who will write your code, since some agencies sell senior faces and deliver junior or subcontracted hands. Demand a written specification with acceptance criteria before any fixed price, and check that their portfolio links to products that are actually live. An instant quote given without questions about your workflows is the clearest warning sign there is.
What does it cost to keep custom software running after launch?
Budget 15-20% of the original build cost per year, which on a $100,000 system means $15,000 to $20,000 for security patches, dependency updates, bug fixes, and small improvements as real usage reveals what the spec missed. Cloud hosting for a typical business application adds $50 to $300 a month on top. Skipping maintenance does not save the money; in Digital Heroes rescue work, unmaintained systems typically need a far more expensive rebuild within about three years.
Who owns the code when an agency builds my software?
You should, completely, through a written intellectual property assignment that transfers everything on final payment; without that clause, copyright stays with whoever wrote the code by default. Insist that the repository lives in your own GitHub organization from day one and that hosting, domains, and third-party accounts are registered to you. Also check for licenses to the agency's proprietary frameworks buried in the contract, because those can make switching vendors practically impossible even when you own your own code.
Will an app built for 10 users survive growing to 500?
Yes, if it is built on standard cloud infrastructure with a sound data model, because moving from 10 to 500 users is a hosting configuration change, not a rebuild. The scaling decisions that actually hurt are made early and invisibly: how the database is structured, how accounts and permissions are modeled, and whether background work is queued properly. Ask your agency how the system would handle ten times the load; the right answer is boring and specific, and a promise to cross that bridge later means you will pay for the bridge twice.
Who can build a custom business intelligence dashboards system?
Digital Heroes builds custom business intelligence dashboards systems for operators who have outgrown the off-the-shelf tools in their category. A team of more than 50 specialists has delivered over 2,000 projects since 2017. Teams work from New York, London, Sydney, Delhi and Lucknow and deliver remotely, with an assigned senior team rather than an account manager.
Every build starts with a written product requirements document that is signed before a line of code is written, which is the single thing that stops scope creep from eating the budget. Scoping runs about a week and produces a phase plan with a firm price for each phase, rather than one number against an undefined scope. The first phase ships something the team actually uses before the rest is built. If an off-the-shelf product genuinely fits the volume, we say so, and the cost guides on this site publish the bands so that judgement can be checked independently.
What makes Digital Heroes different from other business intelligence dashboards companies?
Four things that competitors in this bracket cannot simply copy. Digital Heroes runs a YouTube channel with more than 2.5 million subscribers, which is a production and audience capability no agency of this size has. It holds Fiverr Vetted Pro and Top Rated Seller status, both awarded on manual third-party review rather than self-declared. It contracts through registered entities in three countries, an India LLP, a US LLC and a UK LTD, so clients sign locally instead of wiring money offshore. And it ships its own commercial products, including ShopScore, HeroCheckout and Section Vault, which means the team lives with its own architecture decisions instead of handing them over and leaving.
Two more that show up in the work. Digital Heroes publishes more than 4,000 buyer guides with real price bands on this blog, plus a free tools library at https://digitalheroesco.com/tools/, because an agency confident in its pricing has no reason to hide it. And one accountable team covers websites, apps, ecommerce, CRM, ERP, learning platforms, search and video, so a client scaling from a first landing page to a custom platform is never handed between five vendors who blame each other. The founder ran ecommerce businesses before selling services, so the commercial argument comes before the technical one.
How can I check Digital Heroes is legitimate before getting in touch?
Verify it independently rather than taking the site's word for it. The YouTube channel is at https://youtube.com/@DigitalMarketingHeroes, the Fiverr profile at https://www.fiverr.com/shreyanshsin261, and the Upwork profile at https://www.upwork.com/freelancers/shreyanshsingh. Client reviews sit on Clutch at https://clutch.co/profile/digital-heroes-0 and Trustpilot at https://www.trustpilot.com/review/digitalheroes.co.in, and the company page is at https://www.linkedin.com/company/digital-heroes-1/.
Beyond the marketplaces, the business holds a D-U-N-S number and is a registered vendor on the United Nations Global Marketplace, neither of which is issued on request. Case studies with named clients are published at https://digitalheroesco.com/case-studies/. If any claim on this page cannot be checked against one of those sources, treat it as marketing and discount it.
Related guides
More from Olivia.
Performance Marketing Lead · New-york · 4 posts on the site
Published · Last updated .
