Unlocking Media Insights: Empowering a Not-for-Profit with Smarter Analytics

Quick Summary:

I helped a not-for-profit media organisation transform its analytics capabilities by refining data analysis processes, reducing cloud costs, and empowering staff with cutting-edge tools. By introducing a super-fast analytical query engine (DuckDB), I enabled a leap forward in their ability to extract insights and make data-driven decisions.

I partnered with a medium-sized not-for-profit media organisation specialising in audiovisual content distribution. This collaboration highlights how innovative analytics tools can help organisations optimise operations, reduce costs, and empower their teams.

The Challenge

The client had a solid data foundation using PostgreSQL for structured data storage but struggled with slow and inefficient analytics processes. They needed:

• Faster tools to analyse large datasets
• Better insights into user behaviour and bot traffic
• Cost-effective solutions to optimise cloud infrastructure usage
• Modern tools to empower their team and improve productivity

These challenges are common for medium-sized organisations that have invested in foundational data infrastructure but need to take their analytics capabilities to the next level.

Our Approach

We designed a solution that combined advanced analytics tools, AI-powered innovations, and staff mentoring to address the client’s needs.

1. Supercharging Analytics with DuckDB

To address the client’s lagging analytics performance, we introduced DuckDB, a high-performance analytical query engine. DuckDB seamlessly integrated with their existing PostgreSQL infrastructure and enabled:

• Lightning-fast query performance on large datasets
• Local processing of data without relying on costly cloud-based solutions
• A significant reduction in time required for analysis and reporting

This upgrade allowed the client to extract insights more efficiently while reducing costs—a game-changer for their operations.

2. Data Model & Log Analysis

We developed a refined data model tailored to the client’s needs. This model structured their download log data, enabling:

• Accurate segmentation of bot vs. human traffic
• Deeper insights into audience demographics and behaviours
• Handling of edge cases that had previously gone unaddressed

3. Staff Empowerment

We conducted flexible mentoring sessions to upskill staff in modern analytics workflows. This included:

• Training in tools like Jupyter Notebooks, Pandas, and Quarto for dynamic reporting
• Introducing LLM-powered development environments, which were enthusiastically adopted One team member described the transformation as "light years ahead" of their previous capabilities.

4. Collaborative Platform Development

To showcase the potential for rapid deployment of analytics capabilities, I built a low-code web application using Streamlit. This platform demonstrated how internal teams could quickly access and share insights across the organisation.

5. AI-Powered Innovations

I assisted in evaluating and implementing cutting-edge language detection algorithms using leading AI platforms like Hugging Face and SpeechBrain. These tools enabled advanced audio processing capabilities for speech analysis.

The Results

My work delivered significant benefits for the organisation:

• Lightning-Fast Analytics: The introduction of DuckDB dramatically improved query performance, enabling faster insights from large datasets.
• Cost Efficiency: Optimising cloud usage through smarter data analysis reduced unnecessary infrastructure costs.
• Staff Empowerment: Team members were equipped with state-of-the-art analytics tools, boosting productivity and confidence.
• Actionable Insights: The refined data model provided clear segmentation of download traffic, enabling better decision-making.
• Scalable Analytics: The low-code web application demonstrated how analytical capabilities could be rapidly deployed across teams.

For medium-sized businesses, this project demonstrates how thoughtful use of modern analytics tools can lead to measurable improvements in efficiency, cost savings, and team empowerment.

The Tools Behind the Success

I leveraged a robust technical stack built entirely on mature open-source components:

Client Infrastructure

• Google Cloud Services (GCS): Used for log storage and processing.
• PostgreSQL: A powerful relational database system for structured data storage.

Data Sources

• Log Files: Analysed detailed logs from GCS to extract meaningful patterns.
• Audio Files: Processed audio content for input into language models.

Analytics Tools

• DuckDB: A high-performance SQL-based analytics database that enabled fast queries on large datasets.
• Pandas: A library for advanced data manipulation.
• Jupyter: Interactive notebooks for live code, visualisations, and narrative text.
• Quarto: A tool for creating rich, reproducible analytical reports.

Web Application Framework

• Streamlit: An open-source framework for building interactive web applications.

Modelling Frameworks

• SpeechBrain: A speech toolkit for advanced audio processing.
• Hugging Face: Used to evaluate state-of-the-art language detection algorithms.

What This Means for Medium-Sized Businesses

This project illustrates how medium-sized organisations can benefit from:

1. Supercharging existing infrastructure (like PostgreSQL) with modern tools like DuckDB to enable faster analytics.
2. A custom data model that simplifies complex datasets into actionable insights.
3. Advanced analytics tools that empower internal teams to work more efficiently.
4. AI-powered innovations that open new possibilities for understanding user behaviour or operational trends.
5. Scalable platforms that enable rapid deployment of analytical capabilities across teams.

If your business is looking to optimise its data strategy or empower its teams with modern tools, let’s discuss how I can help you achieve similar results.