Austrian Artificial Intelligence Podcast

Today on the podcast I have to pleasure to talk to Jules Salzinger, Computer Vision Researcher at the Vision & Automation Center of the AIT, the Austrian Institute of Technology. Jules will share with us, his newest research on applying computer vision systems that analyze drone videos to perform remote plant phenotyping. This makes it possible to analyze plants growth, but as well how certain plant decease spreads within a field. We will discuss how the diversity im biology and agriculture makes it challenging to build AI systems that generalize between plants, locations and time. Jules will explain how in their latest research, they focus on performing experiments that provide insights on how to build effective AI systems for agriculture and how to apply them. All of this with the goal to build scalable AI system and to make their application not only possible but efficient and useful. ## TOC 00:00:00 Beginning 00:03:02 Guest Introduction 00:15:04 Supporting Agriculture with AI 00:22:56 Scalable Plant Phenotyping 00:37:33 Paper: TriNet 00:70:10 Major findings ### References - Jules Salzinger: https://www.linkedin.com/in/jules-salzinger/ - VAC: https://www.ait.ac.at/en/about-the-ait/center/center-for-vision-automation-control - https://www.ait.ac.at/en/about-the-ait/center/center-for-vision-automation-control - AI in Agriculture: https://intellias.com/artificial-intelligence-in-agriculture/ - TriNet: Exploring More Affordable and Generalisable Remote Phenotyping with Explainable Deep Models: https://www.mdpi.com/2504-446X/8/8/407

Today I am talking to Philip Winter, researcher at the Medical Imaging group of the VRVis, a research center for virtual realities and visualizations. Philip will explain the benefits and challenges in continual learning and will present his recent paper "PARMESAN: Parameter-Free Memory Search and Transduction for Dense Prediction Tasks". Where he and his colleagues have developed a system that uses a frozen hierarchical feature extractor to build a memory database out of the labeled training data. During inference the system identified training examples similar to the test data and prediction is performed through a combination of parameter-free correspondence matching and message passing based on the closes training datapoints. I hope you enjoy this episode and will find it useful. ## AAIP Community Join our discord server and ask guest directly or discuss related topics with the community. https://discord.gg/5Pj446VKNU ## TOC 00:00:00 Beginning 00:03:04 Guest Introduction 00:06:50 What is continual learning? 00:15:38 Catastrophic forgetting 00:27:36 Paper: Parmesan 00:40:14 Composing Ensembles 00:46:12 How to build memory over time 00:55:37 Limitations of Parmesan ### References Philip Winter - https://www.linkedin.com/in/philip-m-winter-msc-b15679129/ VRVIS - https://www.vrvis.at/ PARMESAN: Parameter-Free Memory Search and Transduction for Dense Prediction Tasks - https://arxiv.org/abs/2403.11743 Continual Learning Survey: https://arxiv.org/pdf/1909.08383

Hello and welcome back to the AAIP This is the second part of my interview with Eldar Kurtic and his research on how to optimiz inference of deep neural networks. In the first part of the interview, we focused on sparsity and how high unstructured sparsity can be achieved without loosing model accuracy on CPU's and in part on GPU's. In this second part of the interview, we are going to focus on quantization. Quantization tries to reduce model size by finding ways to represent the model in numeric representations with less precision while retaining model performance. This means that a model that for example has been trained in a standard 32bit floating point representation is during post training quantization converted to a representation that is only using 8 bits. Reducing the model size to one forth. We will discuss how current quantization method can be applied to quantize model weights down to 4 bits while retaining most of the models performance and why doing so with the models activation is much more tricky. Eldar will explain how current GPU architectures, create two different type of bottlenecks. Memory bound and compute bound scenarios. Where in the case of memory bound situations, the model size causes most of the inference time to be spend in transferring model weights. Exactly in these situations, quantization has its biggest impact and reducing the models size can accelerate inference. Enjoy. ## AAIP Community Join our discord server and ask guest directly or discuss related topics with the community. https://discord.gg/5Pj446VKNU ### References Eldar Kurtic: https://www.linkedin.com/in/eldar-kurti%C4%87-77963b160/ Neural Magic: https://neuralmagic.com/ IST Austria Alistarh Group: https://ist.ac.at/en/research/alistarh-group/

## Summary Today on the show I am talking to Veronika Vishnevskaia. Solution Architect at ONTEC where she specialises in building RAG based Question-Answering systems. Veronika will provide a deep dive into all relevant steps to build a Question-Answering system. Starting from data extraction and transformation, followed by text embedding, chunking and hybrid retrieval to strategies and last but not least methods to mitigate hallucinations of LLMs during the answer creation. ## AAIP Community Join our discord server and ask guest directly or discuss related topics with the community. https://discord.gg/5Pj446VKNU ## TOC 00:00:00 Beginning 00:03:33 Guest Introduction 00:08:51 Building Q/A Systems for businesses 00:16:27 RAG: Data extraction & pre-processing 00:26:08 RAG: Chunking & Embedding 00:36:13 RAG: Information Retrieval 00:48:59 Hallucinations 01:02:21 Future RAG systems ## Sponsors - Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/ - Belichberg GmbH: Software that Saves the Planet: The Future of Energy Begins Here - https://belichberg.com/ ### References Veronika Vishnevskaia - https://www.linkedin.com/in/veronika-vishnevskaia/ Ontec - www.ontec.at Review Hallucination Mitigation Techniques: https://arxiv.org/pdf/2401.01313.pdf Aleph-Alpha: https://aleph-alpha.com/de/technologie/

## Summary At the end of last year, the EU-AI Act was finalized and it spawned many discussions and a lot of doubts about the future of European AI companies. Today on the show Peter Jeitschko, founder of JetHire an AI based recruiting platform that uses Large Language models to help recruiters find and work with candidates, talks about this perspective on the AI-Act. We talk about the impact of the EU AI-Act on their platform, and how it falls into a high-risk use-case under the new regulation. Peter describes how the AI-Act forced them to create their company in the US and what he believes are the downsides of the EU regulation. He describes his experience, that the EU regulations hinder innovation in Austria and Europe and how it increases legal costs and uncertainty, resulting in decision makers shying away in building and applying modern AI systems. I think this episode is valuable for decision makers and founders of AI companies, that are affected by the upcoming AI Act and struggle to make sense of it. ## AAIP Community Join our discord server and ask guest directly or discuss related topics with the community. https://discord.gg/5Pj446VKNU ## TOC 00:00:00 Beginning 00:03:09 Guest Introduction 00:04:45 A founders perspective on the AI Act 00:13:45 JetHire - A recruiting platform affected the the AI Act 00:19:58 Achieving regulatory goals with good engineering 00:35:22 The mismatch between regulations and real world applications 00:48:12 European regulations vs. global AI services ## Sponsors - Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/ - Belichberg GmbH: Software that Saves the Planet: The Future of Energy Begins Here - https://belichberg.com/ ## References Peter Jeitschko - https://www.linkedin.com/in/pjeitschko/ Peter Jeitschko - https://peterjeitschko.com/ JetHire - https://jethire.ai/ https://www.holisticai.com/blog/requirements-for-high-risk-ai-applications-overview-of-regulations

## Summary Hello and welcome back to the Austrian Artificial Intelligence Podcast in 2024. With this episode we start into the third year of the podcast. I am very happy to see that the number of listeners has been growing steadily since the beginning and I want to thank you dear listeners for coming back to the podcast and sharing it with your friends. Gabriel is a Bioinformatician at the Austrian Institute of Technology and is going to explain his work on ensemble methods and their application in the medical domain. For those not familiar with the term, an Ensemble is a combination of individual base models that are combined with the goal to outperform each individual model. So the basic idea is, that one combines multiple models that each have their strength and weaknesses into a single ensemble that in the best case has all the strengths without the weaknesses. We have seen one type of ensemble methods in the past. These where homogeneous ensemble methods like federated learning, where one trains the same algorithm multiple times by multiple parties or different subsets of the data, for performance reasons or in order to combine model weights without sharing the training data. Today, Gabriel will talk about heterogeneous ensembles that are a combination of different models types and their usage in medical applications. He will explain how one can use them to increase the robustness and the accuracy of predictions. We will discuss how to select and create compositions of models, as well how to combine the different predictions of the individual base models in smart ways that improve their accuracy over simply methods like averaging over majority voting. ## AAIP Community Join our discord server and ask guest directly or discuss related topics with the community. https://discord.gg/5Pj446VKNU ## TOC 00:00:00 Beginning 00:03:31 Guest Introduction 00:06:40 Challenges of applying AI in medical applications 00:17:56 Homogeneous Ensemble Methods 00:25:50 Combining base model predictions 00:40:14 Composing Ensembles 00:45:57 Explainability of Ensemble Methods ## Sponsors - Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/ - Belichberg GmbH: Software that Saves the Planet: The Future of Energy Begins Here - https://belichberg.com/ ## References Gabriel Alexander Vignolle - https://www.linkedin.com/in/gabriel-alexander-vignolle-385b141b6/ Publications - https://publications.ait.ac.at/en/persons/gabriel.vignolle Molecular Diagnostics - https://molecular-diagnostics.ait.ac.at/

## Summary I am sure my dear listener, you have heard about genAI that is driven by gigantic foundations models like GPT4 or Stable Diffusion. Generating Texts, Images and even Videos. But what about the 3D Space? What about 3D models that are used as digital twins for the metaverse, for digital cities that are required for self-driving cars to navigate safely in an metropolitan area? What does 3D genAI look like today? Well to give us an insight into what is possible today and what they are building for tomorrow, I have Cristian Duguet on the show. Co-Founder of Nuvo. A company that is focused on building the 3D Foundation models of the future. On the show Cristian will explain to use some of the limitations of current 3D reconstructions from 2D images or sensors. Like issues with shiny surfaces or transparent materials, but as well, how we are currently lacking a good 3D representation for future foundation models. We will discuss how one at the moment has to choose between two conflicting representations. 3D point clouds and 3D meshes. Where point clouds are easy to create from real world object with sensors like the newest IPhone, but are required in high density and therefore compute in order to generate photo realistic renderings. Where 3D meshes on the other hand, made up of polygons in 3D space, need far less points and provide a very easy way to modify and change 3D objects, but are difficult to generate out of sensor data. Cristian explains what is needed to build future foundation models and how they will be able to combine the benefits of point clouds and 3D meshes, as well how Nuvo are working towards building those next generation 3D foundation models. --- ## TOC 00:00:00 Beginning 00:02:27 Guest Introduction 00:06:40 Nuvo 3D 00:17:56 What motivates the demand in 3D models 00:25:50 GenAI for 3D present and future 00:40:14 How Nuvo wants to build 3D foundation models 00:52:24 How will future foundation models look like ## Sponsors - Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/ - Belichberg GmbH: We do digital transformations as your innovation partner - https://belichberg.com/ ## References Cristian Duguet - https://www.linkedin.com/in/cristianduguet/

# Summary Every day you can read and hear about the impact of AI on companies of any industry and size. But are really all business at a stage where they can benefit from the wonders of AI? What about small companies that are not in the tech and dont have the budget to hire data scientists and machine learning engineers. For example, like small retailers of fast moving consumer goods; FMCG in short. that might only have a few stores in a city. They are experts in their field, but lack the personal or infrastructure to have their own AI initiatives. How can they benefit from AI to for example optimize their planning and supply chain? Today on the show I am talking to Eric Weisz, co-founder of Circly, an AI startup that has build a self-service platform for SME's to help them with demand forecasting. Making it possible for none data scientists with little historical data to use their platform and benefit from accurate predictions. On the show we talk about the challenges of building such a one-fits all platform that has to provide value to all kind of different customers without intensive manual configuration and tuning. We talk about how to verify and maintain data quality, and how approaches from federated machine learning can be used to ensure the effective use of prediction model. So that based on the available data and its characteristics models are selected the are efficient to run as a platform provider, reducing costs, while providing highly accurate predictions for customers. ## TOC 00:00:00 Beginning 00:02:42 Guest Introduction 00:05:98 Circly: Demand Prediction for SME's 00:08:05 Demand prediction as an SaaS offering 00:14:37 Ensuring and maintaining data quality 00:26:09 Prediction model selection based on data and efficiency 00:35:04 Federated Machine Learning & Weight sharing 00:39:58 Feature selection and context enrichment ## Sponsors - Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/ - Belichberg GmbH: We do digital transformations as your innovation partner - https://belichberg.com/ ## References Eric Weisz - https://www.linkedin.com/in/ericrweisz-circly-ai-retail/ Circly - https://www.circly.at/