Courses Effectiveness Analysis in a 3rd Party Online Learning Marketplace

Business Challenge:

The client has a partnership with edX, a third-party online learning platform with global appeal. The client used to publish different courses on edX. The syndication team wanted to understand the level of engagement of the global audience with the client’s existing courses published on edX.

Gauging the level of engagement will help to determine the effectiveness of courses among the global audiences and adopt best practices to their native platform. The client approached TurnB to do a course effectiveness analysis.

TurnB Approach:

We identified the data sources for analysis and the following approaches were done:

  • Data were extracted and studied from JSON strings on edX using string functions in SQL to identify available data elements, check completeness & accuracy
  • Exploratory data analysis was performed to identify the right test group for the study on user groups
  • After the analysis, two user groups were identified to perform the study; the users who signed up on edX prior to the window of enrollment into client’s courses and the users who signed up on edX and enrolled for client’s courses
  • Exhaustive analysis was conducted on both user groups; TurnB designed a user engagement index to measure the user group’s engagement with the courses
  • Tested multiple hypotheses for all user segments
  • Evaluated the analyzed outputs and findings were summarized
  • An additional deep-dive analysis was performed to derive insights at a granular level wherever required


Following were the outcomes after the analysis:

  • Instructed the syndication team to introduce certifications, shorter videos and integrated assessments on the client’s native platform
  • Courses published on edX tracked a higher completion which reflected a higher engagement
  • Courses were well structured complemented with shorter videos and integrated assessments
  • The availability of course completion certifications came out as a leading driver of user engagement which further motivated users and instilled the brand value
  • Initiated talks with other global platforms like Coursera, Udemy etc.
  • Facilitated technology adoption
  • More geographical reach and diverse user roles were added to the mix