This is part of Solutions Review’s Premium Content series, a collection of contributed columns written by industry experts in maturing software categories. In this presentation, CelerData Vice President of Strategy Li Kang provides an overview of real-time analytics use cases so you can stay informed.
Building artificial intelligence (AI) solutions has become easier and easier. You don’t need to be a full-fledged data scientist to train a machine learning (ML) model. You don’t need to be a full-fledged programmer to incorporate AI into your business applications. The right low-code tools will take you a long way in developing and producing AI-based applications.
AI based application
Recently, we developed a relatively complex AI application that includes:
- Data collection in the form of user feedback for a simple online game.
- Sentiment Analysis of User Feedback with Different ML and Natural Language Processing (NLP) Models.
- Production and orchestration of all modules together.
AI models are trained and single modules are developed using a low-code platform for data science. The final application is made available to the real world through a dedicated deployment and production platform.
Online game and user feedback
Online game “Guess the Flag” game. The user is presented with a randomly selected flag and has three chances to guess which country it belongs to. Each game session consists of 10 flags. At the end of each session, the user’s success and failure statistics are displayed via a dashboard and the user gets an opportunity to leave a review.
The entire game is implemented as a web application without writing a single line of code. A low-code tool for data science, the KNIME Analytics Platform relies on the concept of “components” to implement interactive web pages. Although originally intended to resemble only dashboards, KNIME components are so simple that we can implement the game’s entire web interface:
- Flag display
- Answer choices
- Final statistics dashboard
- Form of opinion
Sentiment Analysis Application
On all the reviews collected in the last 24 hours, a sentiment analysis application was applied to evaluate the user experience of the game. There are several ways to implement a sentiment analysis application based on available time, costs, estimates and data.
- A solution based on classic NLP rules and statistics is easy to implement but requires some specific language knowledge.
- In the last few years, emphasis has been placed on solutions based on machine learning algorithms. However, in this case, a representative data set should be available to train the algorithm.
- Still in the field of machine learning-based sentiment analysis applications, deep learning models perform better, although more example data is needed for training.
- Recently, pre-trained transformer models have been made available by companies with access to large amounts of text data.
Below is a KNIME workflow that implements a classic NLP-based solution for sentiment analysis. We chose to implement this NLP-based approach for its simplicity. Any of the other procedures mentioned above may be implemented equally well.
Production and Orchestration
Before moving both the flag game and sentiment analysis applications to production, we modularized them as much as possible. One service implemented flag and answer evaluation, one service feedback collection, one service sentiment analysis and another report generation. Each service consists of a logical block and is produced as a self-contained REST service.
Orchestration of all services is achieved through KNIME server features. A KNIME server can schedule module execution, call modules in a cascade after the execution of other modules, and consume services provided by another module using the Call Workflow Service node.
Steps to success
It sounds easy, right? And that’s it. However, there is more to this project than meets the eye. A successful implementation requires several steps:
- Management Acceptance
- Project definition
- Skill set scouting
- Web development
- AI model training and deployment
- Service product
Let’s start from the bottom of the list. The open-source low-code tool we used, the KNIME Analytics platform, took care of web development, AI model training, and model deployment. Its commercial counterpart, KNIME Server, took care of production and orchestration. Both turned out to be the easiest parts of the overall project, mainly thanks to the low-code approach.
Thus, the key to the success of this project is to approach the development phase with a very clear idea of what we want to develop, the resources we can dedicate and how much support we need from management. Nothing really compares to clear thinking. In fact, this is the most difficult part of the project: understanding what needs to be developed and how all the parts can work together. It took more than a few meetings and a few conversations to reach a clear, final vision.
The aim of the project is always to evaluate the feedback left by the users. We started from book reviews, but users should read at least one or two books first and then give feedback. The polling process was very slow. We thought about the songs, but the copyright issue arose. We thought about restaurants, but here we don’t own the data. Then one of us got the idea to build a game. Gamification is a powerful tool to get users to interact. To avoid copyright issues, we created our own game with flags, to which we added a feedback form. Here are some basic UI skills required to make web development as easy to use as possible.
The second part is the evaluation of feedback texts. For that, even a junior data scientist can implement a sufficiently accurate solution based on one of the four sentiment analysis approaches mentioned above.
Finally, the orchestration and production process should be entrusted to a data engineer, someone with an eye for identifying logically discrete tasks and turning them into services.