Big data and advanced analytics

The concept of big data remains elusive to define, but is in this work characterized as data of a varied nature (variety), that arrives in large amounts (volume), and is updated at a high rate (velocity), and due to this cannot be efficiently treated with traditional methods. Using the same starting point, big data problems are calculations that cannot be efficiently solved by traditional methods due to the complexity or speed of growth of the task, or the amount of data needed to solve it. Big data solutions, correspondingly, are systems designed to solve big data problems. Today, more data than ever is produced. This means that the Norwegian Armed Forces will encounter big data problems in the process of translating available data into good decisions. They will therefore need to improve their ability to deal with these issues. The FFI project Information integration for a modern defense has studied big data and advanced analysis, mainly through literature studies on key concepts and technical experimentation to also gain practical experience with technologies associated with these issues. The purpose of this report is to summarize the findings of the project and communicate recommendations in four key areas: Big data solutions, knowledge graphs, neurosymbolic artificial intelligence, and model reuse. The development of technologiesfor dealing with big data problems have long had a big momentum, thanks in large part to large technology actors sharing many high-quality components as open source. This has led to the situation where the supply of technical components that can be used in big data solutions is plentiful. We therefore believe that it is time for the Norwegian Armed Forces to make use of big data technologies, and recommend that: • The Armed Forces set up a big data solution for a selected big data problem in order to gain experience of how to describe such problems, and how solutions should be specified and realized. In this way, the Armed Forces will also gain valuable insight into the competence required to carry out such a process. • The Armed Forces start testing the use of knowledge graphs for a selected analysis task to see if these tools provide value in the form of better and faster analysis results. • The Armed Forces make sure that they have the ability to use techniques in neurosymbolic artificial intelligence for automated processing and analysis of data at an early stage. • The Armed Forces ensure access to updated expertise on the reuse of models for machine learning so that such techniques quickly can be taken advantage of as soon as the need arises. These are measures that will reduce the risk related to acquiring big data solutions, and will in our view ensure that the Norwegian Armed Forces take an important step towards being able to solve their current and future big data problems.