Building a Daily Data Pipeline: Automating SharePoint Excel to SQL

This presentation highlights a project where our team needed to automate the integration of an Excel file stored in SharePoint into SQL, refreshing the data daily to provide estimated enrollments. While this task may seem simple at first glance, it required careful planning and infrastructure setup.
We began by establishing a secure connection between SharePoint and Azure using a linked service. From there, we developed an Azure pipeline with a scheduled trigger that refreshes the data every morning. This automation ensures that the Residential Graduate Programs team has timely and accurate data to assess student engagement and predict likelihood of enrollment.
This project highlights the power of collaboration and cloud-based infrastructure in transforming manual processes into automated solutions.