SnipAId: Review and future prospects
Six months ago, we started the SnipAId project with the goal to support content creators and journalists in generating news snippets for German-language news text using artificial intelligence (AI). Our focus was on evaluating the capabilities of open-source large language models (LLM) for news snippet generation and finding suitable approaches for adaptation.
At the beginning of this project only English open source models were available that were instruction fine-tuned and competitive to GPT-3. First comparable German Instruct models became available just a few weeks ago and have significantly advanced the project. We have tested approaches to adapt both English and German LLMs for the task of news snippet generation.
We evaluated six different news datasets and tested 25 different language models regarding their suitability for news snippet generation. We tried five different approaches to adapt the most promising model candidates, including prompt engineering, prompt tuning, multitask fine tuning, english-centric generation + translation and continued instruction fine tuning.
The most promising approach to model adaptation was continued instruction fine tuning of German LLM IGEL. Few data points and little training time are needed to improve generation capabilities with this approach. We created an instruction dataset with 6 different snippet types and continued training the IGEL model which is a pre-trained, instruction-tuned german version of the BLOOM model.
We also built a semi-automated system, which includes a web app, dockerized backends and a WordPress plugin. The technical stack used for development includes HTML, CSS, JS, PHP, Django, Docker, Netlify, Python, Sanic, Transformers, and Jupyter Notebooks. Serverless GPUs are used for model deployment and operation.
Due to common shortcomings in LLMs such as hallucinations, toxicity and stereotypes that still need to be overcome, these models are best used as an assistive technology where outputs are curated by a knowledgeable human. This project demonstrates that open-source large language models are suitable for news snippet generation in this setting and continued instruction fine-tuning is the most performant approach to further build such a semi-automated system.
You can test the systems at snipaid.tech.
For us, this marks the end of the SnipAId-Project. We encourage interested developers to continue working on the task of News Snippet Generation. Future prospects in this regard could be the extension and improvement of the app and the support for low resource consumption on customer hardware.
TL;DR: Six months of work conclude with a model capable of generating 6 different news snippets in German language and a semi-automated system built around it. A lot of our knowledge has aggregated in this blog. It is now in your hands to carry forth this project.
“SnipAId: Review and future prospects” isthe last part of our series News Snippet Generation. A Learning Journey on Open Source Large Language Models and how to assist journalists in generating headlines, teasers and more in German with AI.