SOURCING VALUE

AUTOMATED DATA PROCESSING SOLUTION FOR SOURCING VALUE

Client

Sourcing Value approached us with significant challenges in managing their data processing tasks. The client was overwhelmed by the variability in Excel data formats and the manual effort required to clean, map, and integrate this data into the correct format and then into their systems. Their primary objective was to find a faster and less manual way to do this so we suggested a way to automate these processes. Our idea would help to reduce the chance of bugs being introduced to their work and the need to troubleshoot, speed up potential operational delays and allow their team to focus on more creative work. They needed a solution that could handle diverse data inputs and outputs programmatically, minimizing the need for manual work by their highly skilled team.

Challenge

Sourcing Value's data management system and process encountered several key challenges. First, data variability was a significant issue for their team. With suppliers and locations submitting data in different, inconsistent formats, it became difficult to standardize any processes and each month they had to repeat the exact same process. This inconsistency led to inefficiencies and errors in how data was processed and employee's depression around performing this menial but important job over and over.

Second, the company was dealing with a large amount of manual effort. Their team had to spend substantial time cleaning, verifying, and organizing data to make sure there was accuracy, which caused delays, required the dev team to help and hampered operational efficiency.

Additionally, the existing AI models used to automate these tasks had their limitations. They struggled to handle the more complex data structures that Sourcing Value worked with, requiring frequent user intervention to correct or adjust the output, thereby reducing the intended value of automation.

Performance constraints were another challenge, particularly with large data sets. Processing times were extended due to the volume of data, which impacted the overall productivity of the team.

Finally, integration posed a challenge. With multiple data sources contributing to the system, bringing everything together into a cohesive and usable output was difficult and time-consuming, further slowing down their workflows.

Strategy

Sourcing Value's current engineering team had limited experience implementing AI solutions or what was possible. We proposed an approach that focused on automating their data processing tasks by developing a "black-box" solution that harnessed Wippy. This solution allowed Sourcing Value to upload diverse Excel files, train AI Agents using their unique business rules, and receive processed Excel files in return without extensive manual work or the need for technical team members to help them. Key technologies and strategies included:

Workflow Automation: Automating repetitive tasks such as file normalization and data transformation.
Dynamic Tool Creation: Developing tailored tools for data extraction, analysis, and transformation.
Programmatic Workflow Compilation: Creating programmatic sequences to handle data transformations seamlessly.
API and EDI Connectors: Ensuring seamless integration of various data points from different sources.

/ Sourcing Value

Solutions

Technologies

Automation
Workflow Management
API Integration
Cloud-based AI Models

Spiral Scout was integral in developing the entire solution, acting as the primary technical team. The team handled everything from design, development, and testing to implementation and integration. The process included:

Initial Analysis: Understanding the client's data challenges and pain points.
Design Phase: Conceptualizing the black-box solution and workflow automation strategies.
Development: Building the tools and integrating them into Wippy.
Testing: Iteratively refining the solution to ensure accuracy and efficiency.
Deployment: Implementing the solution within the client's existing systems and processes.

The solution was seamlessly integrated into Sourcing Value's existing workflows. API and EDI connectors were employed to help with smooth data transfer between Wippy and the client's various systems. The iterative testing and refinement process ensured that the solution fit perfectly with their operational needs.

Black-box Solution: This allowed users to input files and get processed results without manual intervention.
Multi-agent AI System: Wippy used multiple AI models to handle different tasks, providing flexibility and enhancing the accuracy of results.
Cloud Integration: For performance-heavy tasks, cloud-based AI models were utilized to enhance processing speed and accuracy.
AI Model Limitations: The initial AI models struggled with certain complex data formats, which required iterative refinement and debugging.
Data Integration: Integrating disparate data sources into a single, cohesive output was challenging but was overcome with custom-built connectors and data normalization processes.

Results

The implementation of Spiral Scout’s solution resulted in:

Time Reduction: 90% reduction in the time spent on data preparation and processing.
Increased Efficiency: The time to complete went from a month and a half of people time down to 15 minutes for the first time executed and in minutes after that.
Accurate Data Mapping: The solution ensured that data from different suppliers and locations was accurately mapped and formatted with up to 90% accuracy which got them far enough to start.
Scalable Solutions: The programmatic workflows provided a scalable solution that could be adapted to different data processing needs.

The client's goals of reducing manual effort, improving data accuracy, and speeding up processing times were met, if not exceeded. They went from a process that took over a month down to 15 minutes. While not completely automated it got them 90%+ of the way there. The automation and programmatic solutions provided by Wippy aligned perfectly with their operational needs, leading to faster decision-making and more efficient operations.

The initial part of the project took approximately 2 days to set up once the requirements were known and the initial excel files were received from the client.

The solution is expected to continue adding value by providing scalable, automated data processing capabilities that will grow with the client's needs. The programmatic approach allows for easy adaptation to new data formats and sources, ensuring long-term sustainability.

Data and Metrics:

Efficiency Gains: Reduced data processing times by 95%
Accuracy Improvement: Reduced manual errors by 95%
Time saved: The process went from one person month of work to 10 minutes.

Review

“We focused on creating a scalable, automated solution that addressed the client's most pressing challenges, allowing them to focus on their core operations without worrying about data inconsistencies and processing delays.“