As a data scientist with a rich background in digital marketing, I apply data-driven solutions to real-world challenges in media. With experience in independent roles, I've managed end-to-end processes—from raw data to production models—ensuring impactful contributions for my stakeholders.
Evaluation metrics describe how well a model predicts on unseen data, and are used in fine-tuning. In the Python package "implicit", the evaluation module returned errors for common metrics, such as Normalised Discounted Cumulative Gain (NDCG). After some research, the accepted solution was to perform the evaluation manually.
Goal:
Create my own version of the evaluation module.
Considerations for the approach:
I'm particularly interested in the NDCG metric, because it accounts for the position and relevance of a recommended item. Starting with NDCG also allows for the easier addition of other metrics (eg. Precision, Recall, and AUC) because they make use of the components of NDCG.
Result: I chose a NDCG metric as a starting point.
For reference, I'm following this paper, which describes each metric.
Loops have their place in evaluating recommendation outputs, as the metrics are often expressed in iterations and positions that determine the rank of an item. Cython is said to be more optimised at loops compared to regular Python, which may be why it was used in the original evaluation module. However, I found setting up C to be an additional overhead at the prototyping and tuning stages.
Result: I chose a vectorised approach using Numpy arrays, range logic, and SciPy matrices as opposed to loops as an alternative method.
The recommended item needs to be associated with at least two properties (relevance, position in the recommendation). Three methods came to mind:
Result: Produced a new dataframe that indicated the recommendations' relevance as indicated by a positive position value.
Array operations within Pandas was not as straightforward as doing math on int or float values. To regain access to element-wise array operations:
Result: We can then calculate for the NDCG components: Delta function, Discounted Gain, Discounted Cumulative Gain
For the annotated code: Read more
Instead of a fixed K-sized list, IDCG is based on the number of relevant items for each user. This means the the list of relevant items for each user has varying sizes. The size is at most K, but could be less if they have fewer relevant items. Representing the number of relevant items as a list intuitively would call for a jagged array (array values with different sizes); however this is not supported in Numpy as of writing. This leads us to padding the rest of the array as 0 to keep a uniform list size. Zero is used as it doesn't interfere with the necessary calculation.
Result: Padded arrays create the range where we can reuse the Discounted Gain function to calculate for IDCG.
Outcome:
This project resulted in modules that calculated the NDCG from recommendation systems that:
Skills applied:
Digital media rely on user activity to determine marketable audiences. Being able to predict churning users allow for retention efforts to be put in place. In this project, I defined churn as a 30-day streak of a logged-out state, or 30-day streak of non-listenership.
My main considerations were:
The resulting model predicted around 70% of users correctly, with less than 10% being missed churners. Users are then assigned churn predictions based on the model, which informs the retention team for activation.
Main Takeaways:
At the nascency of this digital radio product, I focused on establishing big data pipelines to meet business needs for reporting, visualisation, and machine learning applications. Google Analytics as a platform displays aggregated data; however, the business required analyses involving event-level data. This prompted the need for the Raw Data, which scales up in volume relative to the brand's user activity. With the requirements in mind, I ensured all my contributions were compute-optimised for the large-scale nature of the data. This foundational initiative resulted in high quality useable data for querying, integrating, and modelling.
A number of core challenges arose from the nature of the data and the needs of the business:
Using dataclasses and vectrorised operations in Snowflake Python (SnowPark, similar to PySpark):
The solution needed to be closer to deterministic considering errors would flow downstream, so I went with a nearest neighbour model:
Solution:
Some show_name values were assigned to the wrong brand and a constant 'ERROR' string occupied the user ID field. Solution:
Main Takeaways:
Supply-Side Platforms manage ad space by deciding which instances of web traffic would be good to show ads to; however, not all traffic is profitable, and sending these instances to the auction (real-time bidding) can be costly. The nature of the data and potential target variables were akin to the marketing funnel that had preceding steps that informed the latter. This project aimed to drop unprofitable traffic to save on costs, while retaining as much revenue as possible. As a solution in the digital ad industry, it must deliver the predictions in a fraction of a second.
Given the nature of the data being mostly categorical, the solution made use of categorical features. I explored three ensemble frameworks commonly used for the said data type. Given the funnel-like nature of the data, I created three different architectures:
Additionally, this project had also undergone the production process by creating the accompanying assets: threshold seeker, hyperparameter tuning, modules, tests, a docker image, Airflow DAG and deploying as a cloud service.
Main Takeaways:
Management makes use of spend, revenue, and eCommerce metrics to make marketing decisions. The stakeholders had a preference for tables that could be explored by date range and broken down by date. Another consideration for this project is the fact that the tool will be used by non-technical stakeholders.
The way to accomplish the data needs was by integrating brand assets: Facebook, Instagram, Shopify, Google Ads, and Google Analytics into an interactive reporting tool. With non-technical stakeholders in mind, I have set my own limitations on top of meeting the data needs:
Google assets had native integrations to the selected reporting tool. For the other channels, I used their respective APIs to extract, transform, and store information on a cloud asset. I then integrated all the data on the reporting tool, created calculated fields that combined metrics from different sources, and created the necessary visualisations.
Main takeaways:
In 2021, the New Zealand Police expressed a need to approach call-outs related to mental health issues. To free up the workload of the police force in that regard, it would be good to explore existing trends on mental health-related issues.
Using the NZ Police's Demand and Activity as the main data source, I used R to group and visualise police reports related to mental health based on the activity's classifications, time, and date.
This allowed trends of mental health reports to surface, showing which aspects of social services could be focused on to provide better mental health support.
Main takeaways:
For my first Python project, I wanted to understand popular video topics in a certain niche, and explore Youtube as a data source. One of the largest limitations to the project is the rate limit, which compelled me to consider an ETL process to avoid redundant extraction.
I used Python to extract data, Pandas to perform transformations, and SQLITE to store the results. In its first iteration, the project allowed for visualisations and inspections of overarching trends of videos and the connectedness of channels. In future iterations, I aim to apply machine learning techniques as a form of feature engineering or predictive analytics.
I intend to use this project as a foundational data source for me to explore other data projects and techniques.
Main takeaways:
Covered the introduction to data types, exploratory analyses, encoding, feature engineering, and machine learning options for categorical variables.
As the sole Data Scientist, I elevated the digital radio data from its raw and nascent state to a mature and analytics-ready asset. I pioneered the end-to-end development of scalable data pipelines, implementing methodical transformations to deploy machine learning (ML) models and stakeholder reports. With the large volumes and varieties of data, I developed ML models such as user activity and churn modelling using Python and Snowflake to guide business strategy.
Worked on the ad request optimisation project, which aimed to prevent unprofitable traffic from entering auctions, ultimately improving profit margins through cost savings.
Manage the creation, optimisation, and reporting of Google assets.
Programmatically gather, analyse, and visualise sentiment on the Taniwha using Natural Language Processing techniques.
Provided end-to-end marketing consultancy services ranging from market analyses and brand audits to campaign management and optimisation.
End-to-end management of recruitment marketing campaigns and change management towards the use of the Hubspot Suite for Marketing.
Managed global campaigns from end-to-end and modelled the data structure for the proprietary Ad Serving tool.
Liased high level creative requirements to the team and contributed domain expertise for the proprietary Ad Creation tool.
Managed global campaigns from end-to-end and designed creative materials.
Besides data science work, I enjoy activities like: