Integrated Data Solution

The Vendor is required to provide Integrated Data Systems (IDS) provide the necessary infrastructure to monitor and measure how well government entities serve the people they are designed to help.
- The goal of an IDS is to bring together in one place the capacity to answer questions about the efficacy of the programs used to support citizens. 
- Sharing data across systems holds out promise beyond what can be learned from any one data system. 
- The integration of administrative data systems provides potentially even more compelling information on patterns of multi-system program use, costs and outcomes. 
- Key uses include internal analytics, research, case coordination, development, and direct client outreach to benefit City clients.
- The Uses of Integrated Data:
•    Issue Definition, Strategic, and Operational Planning
•    Internal Analytics
•    Research & Evaluation  
•    Direct Client Outreach & Service Coordination 
•    Grant Development and Support
- Maintains and builds the technical environment required for an integrated data system 
- Develops supporting documentation and processes for the maintenance and use of data 
- Implements security procedures to ensure all data remain secure 
- Supports data governance activities Citywide and for agency data specifically 
- Provides technical assistance to data contributors and data users, within and outside the city 
- Generates raw data files for data users 
- Completes analyses to create aggregate data products and data visualizations 
- Promotes a culture of data sharing within the city and with external partners 
- Coordinates with the Law Department to ensure the legal permissibility of data sharing activities
- Documentation 
•    Revisions of existing technical documents as well as creation and revisions of all operation manuals.  
•    Examples of key documentation include the following:
•    Operations Manuals
•    Technical Manuals 
•    ETL Design and Support
•    Revision of existing models (staging, warehouse, data marts)
•    Disaster recovery (operations and design)
- Performance 
•    Strategic and operational changes to improve overall system performance and scalability.  For performance, the following functions must be performed:
•    Archive old data sets
•    Removal of data sets
•    Index revisions
•    Partition revisions
•    OS server modifications (settings)
•    Database server modifications (settings)
- Data warehouse maintenance and system support 
•    Staging/Warehouse maintenance.
•    OS/Database monitoring and support.
•    Oracle NOS installation and Upgrades
•    Backup/recovery monitoring, revisions and development builds.
•    Provide reports on system operations, execution times and all applicable system metrics.
- Data Integration: Provide end-to-end support for all data requests, including development and refinement of specifications, creation of the codes required for the extracts and the ultimately delivery of the data set(s) extract for new and existing requests (on-going).
- Data Integration Process
•    Extract, Transform, Load (ETL)
•    Data Acquisition Function
o    Both batch and real time capabilities
o    Detection of adds, changes, and deletes in source
o    Accepts many types of data formats
o    Conforms various data types (especially dates)
•    Data Validation and Cleansing Functions
o    Name and address cleansing
o    Metadata-driven checking of codes
•    Exception Logging
o    Automatic recycling of exceptions to accommodate data latency and permit metadata updates to clear exceptions.
o    Maintains referential integrity and logs exceptions to report to source systems
•    Process Controls and Static Maintenance
o    Metadata for tracking and controlling functions
•    Client Matching Function
o    Customized to deal with anomalies in social data
o    Auto rematch when attributes change
•    Transformation from any source to common data stores without any need for database changes
o    Metadata driven Attribute Handling Functions
•    Complete traceability and data lineage to maintain integrity to source data.
o    Maintains complete archive of processed and unprocessed data to enable traceability, reporting and re-run.
•    Intelligent “undo” capability at a file level and complete traceability of any database fixes.
- Budget: $350,000 to $415,000.

- Contract Period/Term: 1 year
- Pre-Proposal Meeting Date: September 11, 2025
- Questions/Inquires Deadline: September 18, 2025