A new big data analysis tool has been developed by computer scientists at the Worcester Polytechnic Institute in the USA, which will help businesses understand all this flood of data!
Computer science professor Elke Rundensteiner of the National Science Foundation is leading a team of data science students building a next-generation data analytics tool, known as SETA. The open-source software will be used to identify patterns in real-time, high-volume data streams and analyze and better understand them. Specifically, SETA could enable large corporations, social media, fraud detection centers, autonomous vehicle networks, governments and other users to use the continuous flow of big data as it flows and transforms into insights. According to the professor, in a world where big data is constantly increasing in volume and speed, real-time streaming data analysis has become extremely important.
Event processing is a way of detecting and analyzing incoming information, such as online purchases, stock price increases and decreases, the length of time users spend on a website, or whether healthcare workers wash their hands before entering patient rooms. Incoming data is all about flagging important events so that an organization can respond in real time. SETA will be able to handle complex requests while providing users with concise information cheaper and faster than ever before.
However, Rundensteiner noted that the most interesting tools for data analysis have not been designed to work with streaming data. Rundensteiner's tools operate on the data as it is generated, allowing even more complex patterns to be observed in real time, so that important decisions can be made quickly. Furthermore, according to the professor, data streams are increasing to such an extent that large enterprises cannot understand the data in real time. If ways to manage live streams are found, then there will be access to new ground for data analysis.
To create new analysis tools, Rundensteiner must first design a new query language, which is used to identify and retrieve patterns in data. The new language will make the tool easier to use, as it will allow users to search for more complex patterns. The professor is also building a new query engine to process sophisticated queries and find the requested patterns. Building this engine is the key to the entire project and will provide all the answers to the desired questions. Soon, the new analysis software will be tested, using datasets and applications that will be provided by a health center and a company that processes financial transactions.
The new technology for big data will therefore be especially useful primarily for the health sector, as it could detect patterns that show the spread of an infection. Generally, there will be the ability to see how problems evolve and where they originate, and simultaneously better tools will be built to provide the necessary answers to the massive wave of incoming information.
