Our company is a startup focused on helping language learning students mastering a new language primarily via voice enabled applications. Currently we work with various OpenAI APIs and just released a language learning partner, voice based assistant you can try for yourself at https://sunnydays.tech/#talk-to-kaki 

*Ideal candidates enjoy an open exploration challenge. Are learning or mastering 1 language in addition to their native language(s). Comfortable using Javascript|Typescript or Python. 

We’re currently investigating a new feature, which you may lay down the groundwork for. The core of the feature is, given an audio file the student was just taught and are willing to repeat out loud, take a recording of their speech and compare against the original audio, while flagging any significant discrepancies where they happen.

The logic may run on the FE or backend. We would like to collaborate with students to research and apply the latest artificial intelligence (AI) and machine learning (ML) techniques to our problem. Altho to be explicit it’s not necessary to apply said techniques if a classic crafted algorithm works well.

The initial language to be considered may be one of: English, Japanese, Spanish, Portuguese. At any rate consider the solution must be able to adapt into multiple mainstream languages.

Input: 2 audio files of the same encoding type (something common like .mp4), a base ideal vs. user recording.

Output: an array of: either points in time, or ranges, where the pronunciation didn’t seem to match input.

Students will develop an AI / ML model related to any of the aforementioned applications.

This will involve several different steps for the students, including:

• Ask clarifying questions
• Investigate research provided by the company along student own research
• Select an open source dataset, or create a small one with examples
• Test open source solutions and academic papers
• Explanation of vocabulary towards the various discrepancies in matching sound
• Highlight steps taken towards the solution, including failed paths
• Produce one working private repository on Github

Bonus steps would include:

• The ideal resulting solution is simple to execute in terms of low dependencies
• Thresholds of how strict evaluation is can be adjusted via 1 or more parameter(s)

Final deliverables should include

• Report of main steps taken in the way of research, consolidation of terminology
• Present a final repository with a working solution and steps to reproduce