Author: Mika Hämäläinen
Metropolia’s AI research strongly featured in an international workshop
Metropolia and the University of Eastern Finland jointly organized the IWCLUL workshop (International Workshop on Computational Linguistics for Uralic Languages), which brought together researchers of Finno-Ugric languages from across Europe. The workshop was held as part of the international ACL community and provided an up-to-date overview of language technology research on Uralic languages, especially in the era of artificial intelligence and large language models. A broad range of Metropolia’s research Metropolia’s AI research was exceptionally well represented at the workshop. Four full papers produced at Metropolia were accepted for the workshop, addressing both pedagogical and language technology topics from multiple perspectives. The paper From NLG Evaluation to Modern Student Assessment in the Era of ChatGPT: The Great Misalignment Problem and Pedagogical Multi-Factor Assessment (P-MFA) examined the impact of artificial intelligence on assessment practices in higher education. The study highlighted the so-called Great Misalignment Problem, where assessment no longer measures what it is intended to measure when students can produce high-quality outputs using generative language models. The paper introduced a new Pedagogical Multi-Factor Assessment (P-MFA) model, which emphasizes the learning process, diverse forms of evidence, and pedagogical transparency rather than single final products. In a paper co-authored with Waseda University, Benchmarking Finnish Lemmatizers across Historical and Contemporary Texts evaluated Finnish lemmatization tools on both contemporary and historical data. The study made use of the Project Gutenberg corpus and, for the first time, included the Trankit tool in a comparison of Finnish lemmatization. A key finding was that Murre preprocessing significantly improves lemmatization results for dialectal and historical texts, while its impact on modern Finnish is minimal. In the image, Aki Morooka is talking about normalization experiments. A timely application of artificial intelligence to foresight was presented in the paper ORACLE: Time-Dependent Recursive Summary Graphs for Foresight on News Data Using LLMs. The study developed a new method in which temporally evolving recursive summary graphs are constructed from news data using large language models. The ORACLE approach enables the analysis of developments and emerging trends in news content by combining temporal structure with language model–based summarization. The fourth paper, co-authored with the University of Helsinki, Evaluating OpenAI GPT Models for Translation of Endangered Uralic Languages: A Comparison of Reasoning and Non-Reasoning Architectures, focused on machine translation for endangered Uralic languages. The study compared reasoning-based and non-reasoning architectures of OpenAI’s GPT models and analyzed their performance on low-resource languages. The results provide valuable insights into which types of language model solutions are best suited for supporting small and endangered languages. Metropolia’s lightning talks: agile openings on topical themes Metropolia’s visibility at the IWCLUL workshop was not limited to full research papers but extended strongly to the lightning talks as well. The lightning talks provided a concise yet substantively rich overview of rapidly developing research directions that are central to language technology for Uralic and other small languages. The lightning talk UralicMCP: Turning LLMs into Experts in Endangered Languages with MCP presented a new Model Context Protocol (MCP)–based extension to the UralicNLP library. The core idea of UralicMCP is to connect large language models with rule-based language technology tools such as a morphological analyzer, inflector, lemmatizer, and dictionaries. This makes it possible for language models to perform NLP tasks even in endangered Uralic languages for which they have little to no training data. Experiments presented in the lightning talk showed that, with MCP, language models can succeed in tasks that would otherwise be impossible for them. Lev Kharlashkin addressed the current state of the Karelian language. The second lightning talk, From Toki Pona to Uralic: A Grammar-Constrained Pipeline for Low-Resource Language Generation, addressed a methodological approach to training language models for low-resource languages. The work used an extremely controlled language such as Toki Pona as a testbed for grammatically guided synthetic data generation. The goal was not Toki Pona itself, but a scalable method that can be transferred to morphologically rich Uralic languages. The lightning talk highlighted how explicit grammatical constraints and validated synthetic data can compensate for the lack of large datasets. The lightning talk Did Karelian Survive the Year? A Small Data Update provided an up-to-date snapshot of the digital vitality of the Karelian language. The talk presented a lightweight yet repeatable data collection process used to analyze Karelian-language online content, particularly in news and article texts. The results showed that Karelian is actively produced online, especially in short news formats, and that even a small but regularly updated dataset can provide meaningful insights into the current state of an endangered language. The fourth Metropolia lightning talk, Evaluating Finnish Dialect Normalization in GPT Models with and without Reasoning, focused on dialect normalization of Finnish using language models. The study compared traditionally fine-tuned GPT-style models with models explicitly equipped with reasoning (chain-of-thought). The results showed that strong pretraining in the Finnish language is more crucial than explicit reasoning, and that reasoning-based fine-tuning can even degrade normalization performance in this task. The lightning talk highlighted important insights into when and how reasoning capabilities should be leveraged in language technology applications. Artur Roos explained what Uralic languages can learn from synthetic languages. From research to practice: AI in support of small languages The IWCLUL workshop highlighted how Metropolia’s AI research brings together theoretical linguistics, practical language technology, and societal impact. Both the full research papers and the lightning contributions demonstrated that large language models are not viewed at Metropolia as standalone, general-purpose solutions, but rather as tools that can be guided, constrained, and complemented with linguistic expertise. The common denominator across Metropolia’s presentations was the reality of endangered languages: limited datasets, rich morphology, and the need for transparent and maintainable solutions. Whether the focus was on rethinking assessment in education, translation of Uralic languages, the digital vitality of Karelian, or normalization of dialectal Finnish, the research emphasized approaches that work even when ready-made data or perfect models are not available. The workshop reinforced Metropolia’s role in the international language technology community as an actor that brings together artificial intelligence, open-source development, and the needs of language communities. At the same time, it demonstrated that research on small languages is not a side track of AI development, but one of its most important testbeds: it is precisely there that the assumptions, limitations, and design choices underlying language models are forced into the open.
Metropolia Develops AI Solutions for Internal Needs
Under the leadership of Development Manager Mika Hämäläinen, Metropolia’s AI team is developing various solutions based on large language models to address the organization’s challenges. The core idea is to solve real problems in a user-centered and agile manner. Since large language models are constantly evolving, there is no longer a need to develop the AI itself — instead, our task is to adopt AI and integrate it into everyday life in an easy-to-use form. Our team currently includes software developers Lev Kharlashkin, Melany Macías Morán and Leo Huovinen, as well as student interns Yehor Tereshchenko, Sheng Tai and Aki Morooka. The tools we have developed are named OpintoHain, Oracle, Grant Writing Assistant, Curriculum Tool, and Moodle AI plugin. OpintoHain OpintoHain was developed as part of a project led by Sonja Saarikivi, with the goal of creating a tool for lifelong learners. The target audience consists of individuals external to Metropolia who wish to update their skills and study at Metropolia — whether by taking a single course or potentially pursuing a suitable Master’s degree. We responded to the challenge by developing a chatbot that understands the course offerings of Metropolia’s Open University. The tool is powered by a RAG (Retrieval-Augmented Generation) model that is familiar with Metropolia’s courses and degree programs. It also includes a multi-agent system with dedicated agents for course and degree recommendations, as well as for study guidance. The OpintoHain tool is available for testing on Metropolia’s website. Oracle Foresight has taken on an increasingly important role at Metropolia — everyone is expected to anticipate future developments, but how? We set out to address this challenge with the Oracle tool, which ingests online content such as news articles and job postings. Based on this input, we can analyze the data using vectorization and clustering techniques. We have already developed methods for identifying weak signals and megatrends, detecting drivers of change, conducting data-driven scenario work and implementing an automated multi-agent version of the Delphi method. The guiding idea is that AI processes foresight data into a ready-to-use format, so that the end user can gain maximum benefit from the insights, even if they have little to no prior knowledge of foresight practices themselves. In putting Oracle into practical use to support real-world applications, we are supported by the foresight working group led by Maani Nyqvist, along with foresight expert Marita Huhtaniemi. Grant Writing Assistant The importance of external funding is growing in the higher education sector. Competition for funding is fierce, and often even strong applications go unfunded. We are developing an AI tool in collaboration with Maarit Haataja, Director of RDI and Project Services, and her team, to enhance Metropolia’s chances of securing external funding. In EU Horizon funding calls in particular, it is crucial that every section of the call for proposals is addressed within the application. Even a strong application can be rejected if it fails to mention even a single sub-point. Grant Writing Assistant automatically analyzes the call for proposals and compares it with the content of the application. Any missing elements are clearly reported to the user, who can then choose to correct them manually or have the AI automatically insert the missing content. The tool is also capable of identifying risks and breaking the project down into work packages. Curriculum Tool Writing curricula is a time-consuming process. Each course-level curriculum should reflect both the goals of sustainable development and Arene competencies. To support this, we developed the Curriculum Tool, which analyzes curricula and visualizes the content of degree programs from the perspectives of Arene competencies and sustainable development. In the development of this tool, Metsälintu Pahkin played a valuable role as a liaison with the degree coordinators. You can read the scientific publication describing the tool for more details. Moodle AI Plugin The Moodle AI Plugin was developed for teachers, enabling them to automatically generate assignments directly in Moodle based on their own course materials. The core idea has been to integrate AI directly into a familiar tool, rather than creating a separate system. Senior Lecturer Tricia Cleland-Silva served as a valuable liaison with the teaching staff during the development process. You can read the scientific publication describing this tool for further insights.
Will artificial intelligence take over?
The rapid development of artificial intelligence has led people to wonder whether AI might one day take power into its own hands. There are plenty of people reassuring us that everything is in human hands and that, ultimately, humans are responsible for everything. But are we? AI may not seize power in the sense of enslaving humanity, but we are already outsourcing power and responsibility to it effectively today. We Worship the Machine Clumsy and poorly designed systems are part of our everyday lives. We already have to spend time clicking buttons in the correct order or remembering to do something in a certain system. And if we fail in these rituals of machine worship, we must sacrifice more working time at the machine’s altar, repeating magic words like “oh, that went wrong,” “hold on, what happened here” and “how did we get there again”. The more time we spend trying to please the machine, the more it heats up its processor – the human is enslaved. Humans must press buttons in the correct order, lest the machine gets angry and punishes them. And how often have you found yourself in a situation where nothing could be done because the machine wouldn’t allow it? These situations have surely happened to many of us. A public transit ticket couldn’t be bought because the app froze, or you missed out on loyalty points at the grocery store because the system didn’t recognize your card. Luckily, a human is ultimately responsible – the same human who can only shrug, because the real power lies with the machine. AI is Already Guiding Us Who gets to decide what truth we believe in? To a large extent, that decision-making power has already been outsourced to artificial intelligences. We often solve our problems by Googling them, but Google doesn't give us answers based on their usefulness or truthfulness – the answers are ranked by AI. Where is the human who takes responsibility when Google's AI feeds us false information or hides things from us? Nowhere – the power lies with AI. AI Easily Learns Which Strings to Pull Large international online stores like Amazon and Temu boldly use AI to steer users toward certain products. Sometimes the cheapest options are hard to find because the smart search has figured out you’re willing to pay more. The responsibility, of course, lies with the person – well, you bought it, didn’t you? We Are Eagerly Handing Over More Power to AI Probably nothing in this text is surprising to anyone; what’s most surprising is the contradiction in our values. The same people who fear AI dominance are often the ones outsourcing more power to AI to make their work easier. One of the funniest examples from the academic world is Turnitin and the automatic checking of essays using AI. We humans are happily giving AI the keys to power Let’s go ahead and let AI decide whose thesis gets approved or rejected, and who gets what grade for an essay. The final responsibility lies with the teacher – who may be incapable of evaluating the reliability of the AI. What could possibly go wrong with this setup?
Does AI only repeat what it has learned?
Artificial intelligence is often criticized with claims that it can only repeat its training data, and therefore always produces plagiarized and average output. Is there any truth to these claims? Claim 1: AI retrieves its answers from a database I’ve encountered this claim often. The idea is that AI retrieves answers from its database, and thus it plagiarizes or fails to find the correct answer. Large language models and image-generating AI models do not, by default, have access to any kind of database. Instead, these models have learned to generate responses independently. The image or poem produced by AI, for example, does not exist as-is in any database. Large language models don’t use databases, but they can be connected to one Today, large language models can indeed be connected to a database. Currently, the most common method for doing this is the so-called RAG model (Retrieval Augmented Generation). In this setup, the AI can retrieve information from a database to support its answer. However, the AI still writes the response itself. Claim 2: AI only produces average answers This claim is more complex, as there are many types of generative AI models. Images are often produced using diffusion models, which begin with a random mess of pixels and gradually transform that noise into a better image. The AI aims to reach some sort of average optimal output, so its tendency is toward the mean. Diffusion models run iteratively – each iteration creates a better image, one that’s also closer to the average. Somewhere between the initial noise and the average optimal lies an iteration where the AI produces good images that haven’t yet converged into uniform, average-looking ones. These images are by no means simply average, even if they inevitably share something with the optimal average. With an update, Adobe Firefly began producing better, though very similar, images What about large language models? They also aim to produce the best possible answer, which often results in an average-like response depending on the prompt. However, large language models have a feature that allows you to adjust the temperature, which influences how average or creative the responses are. At the extremes, adjusting the temperature can make the model generate either extremely bland text or pure nonsensical gibberish. Emergent intelligence The intelligence of large language models is emergent. They can generalize from what they’ve learned to completely new tasks. This simply means that AI models can generate responses to questions they’ve never encountered in their training data. These responses are not merely average repetitions of what’s already been learned, as the AI cannot just mimic its data like a parrot would. Adobe Firefly’s training data guides it so heavily that it cannot generate a wine glass filled to the brim Image-generating AI models do not show the same level of emergent intelligence, as their training data influences their output more heavily than with text models. It can often be nearly impossible to get certain kinds of images from them. Average or not? The claim that AI only produces average responses oversimplifies things. Training data influences AI more or less depending on the model, but that doesn’t mean AI is only capable of producing dull, obvious answers. AI also doesn’t just repeat what it has learned, since it’s trained to provide responses to problems it has never encountered before.
Does artificial intelligence only look into the past?
Lately, an interesting argument has come to my attention: ChatGPT only looks into the past, whereas humans can look into the future. This idea stems from the fact that AI is trained on past data, and for instance, ChatGPT's knowledge of the world is limited to the last date of its training material. However, this does not directly mean that AI is only looking at the past. Machine Learning Always Faces the Unknown The fundamental principle of machine learning has always been to train AI on past data and test it on new, unseen data. This ensures that AI functions as expected even when encountering entirely new information. Machine learning aims to work with new data Before large language models, language technology-based machine learning models often struggled when faced with completely new types of data. For example, AI trained on product reviews did not perform well in identifying positive and negative expressions in literary texts. However, these limitations have been overcome with large language models, as they can generalize their learning to perform multiple different tasks. Do Humans Really Look into the Future Any Better? When we humans encounter something new, we often rely on past knowledge to act. Our own "training data" also ends at the present moment. If we see an unfamiliar furry creature on a leash walking towards us, we logically assume it is a dog. This assumption is based on previous knowledge. If it turns out to be a completely new and unknown animal species, we are surprised by the encounter. Similarly, AI relies on existing knowledge when encountering new things. The difference is that, at present, we do not have AI tools capable of dynamically learning from their experiences and updating themselves. AI will always assume that the furry creature is a dog until its training data includes information that a new pet-friendly species has been discovered. A human, however, would learn this instantly. Foreseeing the Future is Reasoning Just as humans predict the future using reasoning and scenario planning, AI can also predict the future by drawing logical conclusions. Large language models are already capable of reasoning and performing tasks that require thought. AI can therefore look into the future if properly guided with prompts to make predictions. Many AI tools, such as ChatGPT and Perplexity, can also fetch additional information from the web, allowing them to base their reasoning on up-to-date data.
Can AI Be Used to Forecast Change with MLPESTEL?
Dr Khalid Alnajjar and Dr Mika Hämäläinen explored in their MBA thesis the capability of artificial intelligence (AI) to forecast change in the operational environment of companies. For this task, they employed a large language model (LLM) and developed a new theoretical framework called MLPESTEL. The Paradigm Shift that Made Forecasting Possible Traditionally, machine learning (ML) techniques have relied on learning patterns form data for individual tasks. Therefore, such models have been able to formulate predictions only in a very limited application area such as weather forecasting or financial forecasting. However, the dawn of LLMs made it possible for AI to conduct reasoning in domains outside of narrow topics and on textual data instead of numerical data. A Call for a New Framework Although LLMs such as ChatGPT have incredible capabilities in terms of reasoning and answering a variety of prompts, they cannot tackle such a difficult problem as forecasting change by a mere prompt. LLMs can reason, but they need to be given the tools to do so - just like us humans. Furthermore, such a complex task must be split into smaller subproblems. The MLPESTEL framework by Alnajjar & Hämäläinen (2024) The researchers elaborated a new framework called MLPESTEL, which draws its inspiration from PESTEL, a framework traditionally used in business research, and the Ecological Systems Theory, a framework commonly used to understand social development of a child. The former framework is important for the business application area of the research whereas the latter was used to divide each individual PESTEL category into four different subsystems – micro, exo, meso and macro systems. The resulting framework was quite complex for a person to conduct analysis with, but not at all too demanding for an LLM which can easily operate on such a level of complexity. Early Results on AI-based Forecasting The researchers investigated the viability of their method by studying the predictive capabilities of an LLM using the MLPESTEL framework on two international companies: Nokia and Tesla. The method was able to correctly predict the opportunity 5G technology brought to Nokia and the difficulties of global chip shortage that impacted Tesla. The results obtained in the thesis work are promising and serve as a proof of concept. LLMs have reached such a maturity level that they can be used in forecasting tasks. MLPESTEL has extended the theoretical capability of conducting forecasting in the context of operational business environment. This research has paved the road for future studies on LLM-driven forecasting and futures studies. The findings serve as a stepping stone for a more comprehensive platform to be developed at Metropolia University of Applied Sciences.
How AI Can Create Healthier Workplaces (Metropolia at Junction)
Workplaces are built on communication, yet miscommunication is one of the most common and harmful issues faced by organizations today. A lack of transparency can leave employees feeling unheard, while managers struggle to address issues they may not even know exist. This disconnect can lead to frustration, poor mental health, and reduced productivity. At Junction 2024, Europe’s largest hackathon, our team of four from Metropolia's Strategy and Development Services set out to solve this problem with a simple question: What if AI could help make workplaces healthier by bridging communication gaps? With Bubble Burster, we're not just solving communication gaps—we're building workplaces where every voice matters and every issue drives meaningful change Our hackathon idea was an AI tool called Bubble Burster. A platform that combines artificial intelligence, transparency, and actionability to create better communication and well-being in organizations. Miscommunication Hurts Everyone Communication isn’t just about meetings and memos. It’s the thread that ties together an organization’s culture, goals, and mental well-being. But when employees feel unheard or when managers lack visibility into issues, the thread frays. This is how AI could help people understand each other A 2022 study by the World Health Organization (WHO) revealed that poor communication, unresolved conflicts, and lack of employee support are among the leading risks to mental health at work. These risks often go unnoticed because employees hesitate to speak up, or their concerns get buried under vague processes. Bubble Burster as an AI Problem-Solver Bubble Burster empowers both employees and managers by turning communication issues into actionable insights. It works in three core ways: 1. For Employees: A Voice That’s Heard Employees can manually submit workplace concerns or rely on our AI to detect issues automatically. By analyzing virtual meetings and team conversations, the AI raises tickets for potential problems (ranging from excessive workload to workplace harassment) based on guidelines from WHO’s "Risks to Mental Health at Work." Tickets ensure transparency, giving employees confidence that their concerns are visible to managers. The ticketing system serves as a central hub where managers can monitor progress, update statuses, and resolve concerns in real time. The platform doesn’t stop there; employees receive tailored advice powered by an AI chatbot, helping them navigate workplace challenges. The chatbot provides tailored advice to help them navigate their workplace concerns, offering immediate guidance while they wait for management to take action. Figure 2: Daily diary and issue progress tracker 2. For Managers: Clarity and Action Managers rely on clear, actionable data to address workplace issues effectively. Bubble Burster’s ticketing system transforms employee concerns into structured tickets categorized by type, frequency, and department. The AI automatically classifies each ticket, allowing managers to prioritize the most pressing issues and track their resolution in real time. The well-being dashboard provides an overview of company-wide and department-specific trends, empowering managers to identify patterns, allocate resources, and make informed decisions. From acknowledging new issues to resolving ongoing concerns, the system ensures every problem is visible and actionable. Figure 3: Issues list 3. The Bigger Picture: A Healthier Workplace A healthier workplace is achieved by using data, not just non-actionable noise. Bubble Burster provides a well-being dashboard that organizes issues by department, allowing managers to track current employee’s mental health status, and process effectively. Figure 4: Company health represented in dashboards Our Hackathon Journey Bubble Burster was created during Junction 2024, a 48-hour sprint that challenged us to think fast, code faster, and build something impactful. Over two days and more than 15 hours of coding per day, our team transformed an ambitious idea into a working prototype by leveraging AI into the workplace. Representing Metropolia's Strategy and Development Services, where we specialize in applying AI to solve real-world problems, we brought together our expertise in AI and workplace dynamics. Just as we’ve worked to bring AI closer to teachers through our AI-powered Moodle plugin, we aimed to make AI a valuable tool for improving communication and well-being in organizations. The collaboration between Mika Hämäläinen, Leo Huovinen, Lev Kharlashkin, and Melany Macías, showcased the power of teamwork under pressure. By combining our skills and experience, we built a platform that has the potential to create healthier, more transparent workplaces. Why Bubble Burster Matters Our goal with Bubble Burster isn’t just to solve communication problems—it’s to create a healthier, more transparent workplace where employees feel valued, and managers can take meaningful action. By leveraging AI, we have built a system that: Detects problems employees may not voice themselves. Provides actionable data and advice for both employees and managers. Fosters trust, accountability, and well-being across teams. We hope Bubble Burster inspires organizations to view communication as a cornerstone of mental health and productivity, and this is just the beginning. As we refine the platform, we envision deeper integrations with tools like Slack, Teams, and Zoom, making it even easier for organizations to adopt. By expanding its capabilities, we aim to create workplaces where everyone feels heard, supported, and empowered.
AI research on languages related to Finnish was presented at Metropolia
The 9th International Workshop on Computational Linguistics for Uralic Languages, or more familiarly IWCLUL, was held at Metropolia. The event brought a large group of international researchers to the Arabia campus, where they presented their language technology research related to Uralic languages, which are languages related to Finnish. The challenge of being endangered Of the Uralic languages, only Finnish, Estonian and Hungarian are majority languages with official state support. The other Uralic languages are more or less endangered. The number of speakers varies from Meadow Mari with 360,000 speakers and Erzya with 300,000 speakers, to Skolt Sámi with 300 speakers and Ume Sámi with just 5 speakers. Some languages no longer have native speakers at all. However, hope is not lost even for these languages, as Valts Ernštreits, the director of the Livonian Institute, often remarks: "Every time the last speaker of Livonian is believed to have died, a new last speaker emerges from some cottage." Jack Rueter reminded us of the importance of popular culture also in the context of endangered languages. Modern language technology requires a lot of data, which makes AI development for smaller languages more challenging. Often, there is little to no data available, and it tends to have significant variation. Spelling rules are often not as clearly defined or deeply ingrained in the speakers’ habits as they are for major languages. Large language models sparked discussion. Large language models like ChatGPT do not currently support any small Uralic language. However, researchers have devised methods to elicit responses from these models by carefully crafting prompts. In addition to my own presentation, both Flammie Pirinen and Niko Partanen reported the results of their research. IWCLUL was organized through volunteer efforts. In the picture, Lev Kharlashkin is inviting the next speaker to the stage. The problem with large language models, even for Finnish, Estonian and Hungarian, is that they split words into smaller units, tokens, based on the English language. Iaroslav Chelombitko and Aleksei Dorkin had proposed solutions for this issue. Metropolia's values on display The work done at Metropolia in the fields of sustainable development and artificial intelligence was also highlighted at the event. Melany Macías presented our research, in which AI learns to predict sustainable development goals in Finnish based on English-language data. Melany Macías presented the accuracy of predicting sustainable development goals.
What are embeddings produced by LLMs?
An ordinary user interacts with large language models, like ChatGPT, by writing prompts through the user interface. In addition to this, large language models offer another functionality for technically skilled users – the creation of embeddings based on text. But what exactly are these embeddings, and what are they used for? Meaning of text in vectors When a large language model is given some text to embed, it produces a vector as a result. A vector is a list of numbers that may not be immediately interpretable to the human eye, but it enables the exploration of the text's meaning through mathematical methods. These vectors produced by the language model are called embeddings. UralicNLP Python library provides tools for embedding text using different language models. Here is an example of how text can be embedded with OpenAI's model using UralicNLP. from uralicNLP.llm import get_llmllm = get_llm("chatgpt", "VAIHDA TÄHÄN API-AVAIMESI", model="text-embedding-3-small")llm.embed("Teksti, jonka haluat upottaa")>>[-0.1803697, 1.1973963, 0.5283669, 1.5049516, -0.27077377...] As seen in the example, the result of an embedding is a list of numbers. These numbers represent the meaning of the text and can be used to compare the similarity of texts through mathematical methods. What are the benefits of embeddings? With embeddings, large volumes of text can be stored in a vector database for quick retrieval. This means database searches are based on meaning rather than character strings. The most common use case for such vector databases currently is the RAG model. RAG stands for Retrieval-Augmented Generation, which refers to a process where a large language model is provided with not just the user prompt but also source material to help generate a response. Retrieving the source material involves using embeddings to find documents relevant to the user’s input from a vector database. For example, Metropolia’s own Mikro-Mikko operates based on this principle. Embeddings can also be used to automatically group text documents into clusters of similar texts. This can be done with UralicNLP as follows. from uralicNLP.llm import get_llmfrom uralicNLP import semanticsllm = get_llm("chatgpt", "VAIHDA TÄHÄN API-AVAIMESI", model="text-embedding-3-small")texts = [“koirat on hauskoja”, “autot ajaa nopeasti”, “kissat leikkii keskenään”, “rekat ajaa kaupungista toiseen”]semantics.cluster(texts, llm)>>[[“koirat on hauskoja”, “kissat leikkii keskenään”], [“autot ajaa nopeasti”, “rekat ajaa kaupungista toiseen”]] The result is that texts are grouped into clusters of similar texts using embeddings and calculating their similarity. Does the model matter when embedding? Embeddings can be generated using both commercial large language models and open-source language models. When choosing a model, it’s important to remember that embeddings are not compatible across models. For example, you cannot create some embeddings with OpenAI's GPT-4 and others with an open-source LLaMA model and expect them to work together. Each model has learned its own representation of meaning from its training data, so the numerical content of the embeddings varies between models. When choosing a model, it's important to consider the cost of the model, the languages it supports, and its context window. Larger models can accommodate a large amount of text within the context window, allowing for a single embedding of an entire text. Smaller models require the text to be split into segments. This technical limitation can be significant depending on how the embeddings are intended to be used. Not all models support all languages. If a language model produces poor Finnish responses to prompts, it likely does not understand Finnish very well. Consequently, embeddings generated for Finnish text may not capture the meaning accurately enough.
Did Generative AI Replace us in Creative Work?
Generative AI models like ChatGPT and Midjourney are capable of producing creative text and images faster than humans. So, why do we still need poets if ChatGPT can generate as many poems as one could ever request? And what about illustrators? Midjourney can produce stunning images even in the hands of an amateur prompt writer. Is it really the case that human creativity can be replaced by machines? What is Creativity? The relationship between human and computer creativity has been studied for a long time. Even long before the era of ChatGPT, researchers were developing AI models that could generate stories, music, and images. This field of study is called computational creativity. The interplay between human and machine creativity has long captivated researchers in this domain. Boden is one of the theorists in computational creativity. According to her, creativity can be divided into two categories: exploratory creativity and transformational creativity. This distinction is crucial when we aim to understand the limits of computational creativity. Artificial Intelligence Exploring Creative Possibilities Currently, all major AI models are capable of exploratory creativity. This means that AI models operate within a defined creative search space, exploring the possibilities within it. For example, if we consider an image made up of 512x512 pixels, where each pixel must represent one of a predetermined set of colors from a color palette, it is clear that the number of possible images is finite. An AI model capable of exploratory creativity can thus generate images only within these rules — these rules define and limit the creative search space. AI exploring suitable pandas from a constrained search space. The images were created by Khalid Alnajjar. The situation is even more constrained in reality. No generative AI model can produce all possible images that could exist in a 512x512 size. AI's functionality is also limited by the data it has been trained on. If you ask an AI for an image of a panda, it will not generate all possible panda images, but only those that align with the understanding of a panda it has learned from the data. In creativity, boundaries are made to be broken Exploratory creativity is clearly confined to a specific search space. But what kind of creativity is shackled by limitations? After all, boundaries are made to be broken! Boundary-breaking creativity is transformational creativity, as it transforms the boundaries of the search space itself. If I am given an A4 sheet of paper and tasked with drawing a house, my creativity is confined to creating a two-dimensional image. As a human, I can pick up the paper and fold it into a house — this is transformational creativity, as I have altered the search space. The third dimension enables the creation of entirely different houses from the A4 paper compared to the original two-dimensional sheet. AI is not yet capable of breaking boundaries. None of the popular AI tools can transform their search space and decide, for example, that instead of creating an image with pixels, they will paint it with a brush. Nor can they spontaneously try a new style that hasn't been taught to them in the training data. The role of humans remains essential AI can produce impressive and creative outputs, but it remains bound by its patterns. The creativity of AI stays within the constraints of its task and training data, and it cannot generate anything beyond the frameworks it has been given. This does not mean that AI is not creative or that it is inherently poor at creative tasks. It simply means that its creativity has not yet reached the level of human creativity. Therefore, there will still be a need for human creativity in the future, at least until we develop a categorically different kind of AI.