The Indian Institute of Technology Madras (IIT-M) on Tuesday said its faculty and AI4Bharat have developed artificial intelligence (AI) models and datasets to process texts in 11 Indian languages.
AI4Bharat is a platform for building AI solutions for problems of relevance to India.
According to IIT-M, its researchers and AI4Bharat released AI models and datasets for the following languages: Tamil, Hindi, Malayalam, Telugu, Kannada, Punjabi, Bengali, Odia, Assamese, Gujarati, and Marathi.
The multilingual AI models and datasets developed through this initiative will provide the essential building blocks to students, faculty, startups and industry to work on the Indian language tools and push the frontiers of technology.
The faculty have made these cutting-edge resources open-source and completely free of cost, which can be accessed by anyone.
These models are freely available and can be downloaded from a Github repository (https://indicnlp.ai4bharat.org/).
Elaborating on this initiative, Mitesh M. Khapra, Assistant Professor, Department of Computer Science and Engineering, said: “We have a very rich diversity of languages in our country. As we move towards a digital economy, it is important that our languages find a space online. This requires a lot of innovation in creating input tools, datasets, and AI models for Indian languages.”
For example, imagine a learner who posts a question on an e-learning platform in Tamil or Hindi or any other numerous Indian regional languages.
There is a need for tools that can automatically process such questions written in the Indian languages and classify them into specific topics.
“While such tools are available for English and other foreign languages, there are hardly any tools for Indian languages and this is the critical gap that we are trying to address through this initiative. These models are available free of cost as we want the entire country to benefit from them,” added Khapra.
AI4Bharat is an initiative co-founded by Khapra and Pratyush Kumar from IIT Madras and works to solve India specific problems in a community-driven, open-sourced manner.
Speaking about the technology behind this initiative, Anoop Kunchukuttan, a volunteer at AI4Bharat and the lead researcher on this project, said: “We have an urgent responsibility to take the rapid advances of AI and make them accessible to the common man. One way of achieving this is to improve interactions between humans and machines. That is where the field of Natural Language Processing (NLP) comes in. NLP is a branch of AI that deals with the interaction between computers and humans using natural language.”
For the past one year, a team of researchers comprising students, faculty and volunteers from IIT Madras and AI4Bharat worked on collecting data and training powerful models for processing text written in Indian languages.
The models take advantage of the similarities between Indian languages to make efficient use of data.
