Data Crawler Software Replacement

The vendor is required to data crawler replacement consists of items detailed in the requirements within this document which includes the following:
•    Provide a data analytics and data crawler tool for agency that can scan both structured and unstructured data repositories. 
•    Enable more useful searching and locating of information to reduce rot and for litigation matters and open records requests.
•    Enable agency information governance and management (IGM) to gain better understanding of information types in agency repositories. 
•    Enable application of appropriate measures to protect sensitive, confidential, or privacy-related information, reducing risk to agency. 
•    Assist in creating an accurate data map of agency systems.
- Functional requirements
a. File type recognition
•    The solution shall provide the ability to identify and recognize different file types.
b. File actions
•    The solution shall allow users to perform actions including migrating, quarantining, deleting files, and notifying people there are files for review.
c. Metrics collection
•    The solution shall collect metrics related to file removal and deletion including:
o    Total number of files present and removed from a repository
o    Volume (space in GB) changes over time within a repository
d. Automatic triggers
•    The solution shall have built-in automation to include notifications and automatic process completion:
o    Files marked for quarantine
o    X days pass (trigger)
o    Email automatically sent to system user indicating x days passed
o    Solution administrator deletes files using system interface
o    Files marked for quarantine and x days selected for duration
o    X days pass (trigger)
o    Solution deletes files
o    List of rot created
o    List of potential rot sent to outside department
o    Outside department does not respond to report after x days (trigger)
o    Email reminder automatically sent to outside department
o    Additional y days pass (second trigger)
o    Email reminder automatically sent to outside department and additional email sent to solution administrator).
o    Migration of files should include refresh of dashboard details
e. Sorting and moving
•    The solution shall allow selection of files after sorting and filtering and move them to a quarantine repository.
f. Clear task labels
•    The solution shall use clear user interface labels for processes and tasks.
g. Confirmation prompt
•    Solution shall provide confirmation prompts for actions such as deletion or migration.
h. Consistent report format
•    The solution shall provide reports that are exported are in the same format (excel or other) as required for upload, if solution requires uploads for any purpose.
i. Data identification
•    The solution shall identify redundant, obsolete, trivial files, and non-records and give a subcategory for these such as "obsolete-past retention".
j. Log actionability
•    The solution shall provide the ability to view action logs and generate log reports (retained indefinitely) for auditing who used the solution in what way on what files when (including access, review, classification, and deletion activities).
k. Saved filters and searches
•    The solution shall have the ability to save filters and searches for routine searches including rot candidates, PII (personally identifiable information), phi (personal health information), etc.
l. ECM integration
•    The solution shall integrate with enterprise content management systems shared network drives, the MS 365 environment, and other repositories.
m. Multi
•    The solution shall allow work on more than one task such as running reports while system is migrating files to quarantine.
n. Outlook integration
•    The solution shall integrate with microsoft outlook to add reminders.
o. Deletion review reports
•    The solution shall generate reports for file deletion review, including files to be deleted, files to be retained, and files for further discussion.
p. Folder
•    The solution shall have the ability to report on both individual folders as well as whole repositories.
q. Role
•    Solution shall provide role-based access to the system (e.g., igm users, legal users, read only access, basic administration, and advanced administration). 
•    The solution will not allow administrative rights to files and folders of a repository to be gained for users through the solution.
r. External review reports
•    The solution shall have the ability to provide a descriptive file report to employees in other departments for review and response.
s. Comprehensive data scanning
•    Laserfiche cloud (estimated 10tb)
•    ASITE construction management software (cloud-based)
•    o365 online environment (estimated 8tb in teams alone) workday (cloud-based) oracle applications
•    Databases such as MySQL, PostgreSQL
•    Salesforce (cloud)
•    Service now (cloud)
•    Share point (online)
•    Network file shares (estimated 7.5tb in largest one) and personal drives
•    Audio and video files (retained on file shares)
•    Official agency social media sites: Facebook, twitter/x, LinkedIn, Instagram, YouTube
•    Agency official website.

- Contract Period/Term: 2 years
- Pre-proposal briefing (Non-Mandatory) Date: July 29, 2025
- Questions/Inquires Deadline: August 4, 2025