HeatWave + GenAI: Oracle’sAI-Powered MySQL Just Changed the Game

Table of Contents   

1. Welcome & MySQL Background

2. HeatWave Functionality and Ease of Use

3. HeatWave GenAI Overview and Key Features
   

4. Practical Use Cases and Applications of HeatWave GenAI

5. Methods of Querying with HeatWave GenAI

6. SQL Examples and Practical Demonstrations

7. Live Demo and Closing Remark
   

Transcript

Welcome & SQL Background

So, welcome everyone. I've got Tony Darnell ready to talk to you about HeatWave more Genai talk. So, I will turn it over to you, Tony. And we got about 25 minutes. We'll have a little bit of time for questions at the end. Um, you can definitely ask them in the meantime through the Q&A there on the side and we'll we'll get to them as we can.

All right, great. Thank you very much. Um, I'm Tony Darnell, principal cloud architect with Oracle. I specialize in Heatwave and MySQL. And so today I'm I'm going to be talking about Heatwave Gen AI.

And I would think that most people here have heard of MySQL, but let me just give you a quick background summary. MySQL is actually celebrating its 30th anniversary this year, and it is the second most popular database right behind Oracle. uh but it is the most popular open-source database and MySQL is well known for OLTP online transaction processing and that's really where it shines. Uh but MySQL hasn't really been known for OLAP or analytics and that is until uh heatwave came along.

So some of you may know MySQL some of you may not be familiar with heatwave. So let me just give you a quick overview before we get into the genai piece.

MySQL Heatwave is a fully managed cloud service that accelerates MySQL database query performance particularly for analytics and mixed workloads.

On on the left we have the what I'm going to call the regular MySQL which uses the NODB storage engine. Um, and NODB is ASI compliant and it stores the data on disk.

Uh, oops, sorry about that. Um, in the middle here we have the Heatwave cluster which runs on separate hardware from the regular MySQL piece on the left. Heatwave is an in-memory columner database and it uses an in-memory query accelerator that allows for faster processing of large data sets and it reduces the need for a separate data warehousing solution and it also eliminates any ETL processes.

So companies that store their transactional data in MySQL, you know, when they want to run analytics uh on the data, they usually have to export the data and then import it into their, you know, their data warehouse in order to run these analytical queries, you know, again because regular MySQL uh doesn't perform well with very complex or or analytical queries.

So um you can have your OLTP data in MySQL and then you can load it into heatwave and once you do the initial load uh when the data is updated in the regular MySQL piece uh those changes are made in Heatwave. So your data stays up to date in both places. You can then send your queries both OOLTP and OOLAP to MySQL and the optimizer will determine if that query should be run against the NODB storage engine or against heatwave. So again there's no ETL. So you this allows you to run analytics on live data.

And then on the on the right here we have files in object storage. And you can also load these files into heatwave and then run queries against that unstructured data stored in the files. And now when you do that, the data stays in the heatwave cluster and it is not copied over to regular MySQL. So all of your data is combined in one system.

HeatWave Functionality and Ease of Use

Excuse me. So overall, HeatWave is is very easy to use just just like MySQL. And loading your data into Heatwave from NODB only requires two commands. Uh the first command is is just an alter table statement. It tells MySQL that besides using the NODB storage engine um for a particular table, you're going to tell it I want to use a secondary storage engine which is called rapid. The second command then loads the data into HeatWave. Now once again once you've executed the second command remember any changes to the data in NODB you know a regular MySQL is replicated to the in-memory heatwave cluster and again once the data is in HeatWave you simply just run your analytical queries and the MySQL optimizer will determine uh which storage engine to use. And then when you're finished with your analytics, let's say you only need to do analytics or you need to run your reports just once a month, you can turn the HeatWave cluster off if you want to save some money and um when you turn it back on, you just issue that second command again and and reload the data. And so that's that's the basics of Heatwave, you know, from a 30,000 foot view.

Now besides regular MySQL and heatwave which is the third section here, HeatWave also gives you lakehouse and auto machine learning capabilities uh as well as the genai features I'll be talking about uh next. But you can see that you don't need separate applications to do any or or all of this. Everything you you see here is in one database system.

HeatWave GenAI Overview and Key Features

Uh I'm not going to talk to you about why you should use Genai. Most of you probably already know the benefits, so I'm just going to get into talking about Heatwave Genai. Heatwave Genai is Oracle's generative AI service integrated into the Heatwave database platform and it is designed to simplify AI development or AI application development.

Heatwave genai has built-in database LLMs uh like llama 3 and mistrol and so this enables data search context generation and rag or retrieval augmented generation without requiring an external AI service or LLM and so this reduces both the complexity and the cost with our automated vector store this automatically generates embeddings for unstructured data And it streamlines semantic search and eliminates the need for separate vector databases.

For scale out vector processing, you can have parallel processing across up to 512 nodes. And so this ensures fast, accurate semantic searches. And heatwave outperforms our competitors. It is 30 times faster than snowflake, 18 times faster than Google query and 15 times faster than data bricks. and it is much less expensive than all three.

Heatwave chat is a natural language interface for contextual human-like conversations with your data and it follows or it supports follow-up questions and source verification.

But the best benefit is that you get all of this at no additional cost to your Heatwave service. The end database LLMs and embedding models are included at no additional charge and GPU provisioning is unnecessary which lowers your operating cost. Heatwave runs on Oracle cloud, AWS and Azure which gives you you know consistent performance and flexibility especially if you have a multicloud strategy.

Practical Use Cases and Applications of HeatWave GenAI

So generative AI with heatwave enables a lot of new use cases. You can use heatwave chat uh to easily retrieve documents and have conversations with your unstructured data using natural language. Uh the context of the questions is preserved to enable a sort of a human-like conversation with follow-up questions. You can use the LLMs to generate reports and you know summarize content in your data. You can perform rag or retrieval augmented generation using the heatwave vector store as well as similarity searches and um or similarity searches on your unstructured content, you know, to find say a similar sentence or maybe a picture. And finally, another very interesting and beneficial use case is the ability to combine machine learning and genai in your apps. And this is very easy with Heatwave as it's all it's all built in. So, this enables you to save time and deliver more value to customers by combining the machine learning output in the LLMs with the vector store. So, you're going to get more accurate and faster results using GNAI on data that's been filtered by machine learning.

So, let's look at a few examples.

So for content generation, you can use the in database LLMs to help you know generate or summarize content based upon your unstructured documents. Uh you can ask questions in natural language via an application and the LLM will process the request and deliver the content. So in this example, we are asking for a summary of this solution brief which comes from documents that we've loaded into heatwave and then JI will provide that summary for us.

Uh a similarity search focuses on finding related content based on semantics. uh solity similarity search goes beyond you know simple keyword searches by you know considering the underlying meaning instead of only searching for applied tags. In this example, a lawyer wants to quickly identify a potentially problematic clause in their contracts. So the person, you know, might ask in what contracts do we have this particular sentence? And then Jenai will provide them with a listing of those contracts.

For rag, you know, chat bots can use rag for example to, you know, help answer employees questions about internal company policies. You know, your internal documents, you know, detail in these policies or or they're stored as embeddings in the HeatWave vector store. And then for a given uh user query, the vector store helps to identify the the most similar documents by performing a similarity search against the stored embeddings. And then these documents are used to augment the prompt given to the LLM so that it provides an accurate answer.

And then like I mentioned earlier, you can enhance rag with the built-in machine learning. So your developers can build applications leveraging the combined power of heatwave's built-in machine learning, generative AI and vector store to deliver, you know, more personalized recommendations. So in this example, the application is using the HeatWave AutoML recommener system to recommend restaurants based upon the users's preferences or maybe what the user has previously ordered. And then with HeatWave Vexor Store, the application can additionally search through restaurants uh menus which are stored as PDFs to suggest, you know, maybe specific dishes. And this is going to provide greater value to your customers.

Uh here's another example with analysis generation. So in this example uh a manufacturing company also combines genai with machine learning for predictive maintenance. uh engineers can use the heatwave autoML to help automatically produce a report of say anomalous production logs and then heatwave genai helps to rapidly determine the root cause of the issue by simply asking a question in natural language instead of you know manually analyzing all of the different log files.

Methods of Querying with HeatWave GenAI

And there are three different ways to ask questions with heatwave genai. The first is with rag. Uh and with rag, you ask natural language questions based solely upon the data obtained from documents that you've loaded into heatwave. So it only searches your data.

With generate, you can ask questions based upon the training data that is stored in the LLM that you select. So it doesn't touch your data. It only uses data from the LLM.

And then for Heatwave chat that uses both rag and generate. So when you ask it a question, if it doesn't find the answer in your data, it will then use the training data from the LLM to help generate uh the answer.

Heatwave has two built-in LLMs, Mistrol and Llama 3. But these LLMs run on CPUs and not GPUs. So they are less expensive, but they are a little slower than the responses that you normally would get from other, you know, uh, chat or genai services that you've probably tried or used. But again, there's no additional cost to use the built-in LLMs.

If you need faster responses or you want a larger LLM, you can use the LLM LLMs from Oracle's Generative AI service. And yes, there is a a small fee associated with using these services and but these LLMs run on GPUs, so your response time is is much quicker.

SQL Examples and Practical Demonstrations

So, I'm going to show you just a couple of examples. I don't want to get too technical here on how easy it is to use simple SQL routines for heatwave jai. So I went to NASA's website and I downloaded about a hundred or more uh electronic books or ebooks in PDF format and then I've loaded those into heatwave. So my data set is basically all about NASA.

So the first command you see is calling the heatwave chat routine which uses again both rag and generate. So I'm going to call the routine and ask my question in natural language. What is the primary function of NASA? JI then takes the question does a vector search sends it to the LLM and it generates the response that you see here.

Um, I do have the ability to change the parameters for Heatwave chat with the chat options variable. And in this example, I'm specifying the database and table name. And I'm changing the LLM that I want to use to use one of Oracle's uh generative AI LLMs. And then the history length. I'm only going to allow just one question and no follow-up questions. And then after setting the chat options, I can ask my question, you know, when was the last space shuttle mission STS135? And then it goes and searches the LLM or I'm sorry, searches my data using the Oracle LLM and then it provides me with the answer.

Uh I can also do the same for RAG by, you know, customizing the different parameters and then I can ask it my question. And I apologize the text on this is probably a little bit small but uh the output in this in this case is generated as a JSON document and then I can u you know output the entire document and what I've shown you here you know the output is truncated because it wouldn't all fit on this on the the screen here but you can see the format that the response come back comes back as is actually the top three answers.

Um, but if I want to, you know, drill down and say I just want the the first answer, I don't care about the top three. I don't care about the distance between the answers. Um, then I can go in and just use a different SQL function and bring out just the first response to my question and it gives me the answer here.

So, and then with generate I can again customize the parameters as well. But remember, generate only uses the training data in the L in the LLM. So, I can ask it a a non-NASA question. Um, because I know this particular answer is not going to be in the documents that I've loaded. So, I'm going to ask it to give me a recipe for fried chicken. And you you've got the answer shown here. Now, I haven't tested this recipe, but just looking at the ingredients, I think it they're probably pretty spot-on here.

Live Demo and Closing Remarks

So, I do want to give you just a quick demo. Um, for the demo, one of our engineers built a simple website which allows me to test Heatwave Gai and all the different uh flavors of LLMs.

Uh, but you could also use MySQL shell. You could use uh the MySQL shell plugin for VS Code, you know, or your favorite scripting or programming language. So again, I went to NASA's website, downloaded about a 100 ebooks, loaded them into vector storage where first I copied them to a bucket in OCI, so a storage bucket. I loaded the the bucket uh with a one simple command that you see here. So again, very easy to set up and to use. Call SIS vector store load. I give it the location of the documents or tell it what uh database I want to use, hit return and it loads the document. It vectorizes it automatically for me. It's just very simple and easy to use.

And I think with that, yeah, let me go to let me switch screens here and pull up my demo.

If I can find it. There we go.

All right. Hopefully you can see the the demo here. There we go. All right. So, what I what we built here and we can probably share this code with you if you are interested. Um, we have uh the ability to basically just test all the different functionality of Genai. I've got my different LLMs here. The first two are the built-in ones that come free, no additional charge. The next five are what's available through the Oracle Generative AI service. So, I'm going to select one here. I'll select the Llama 3.3. It has 70 uh 70 billion Hold on. Sorry. um in the LLM and then I'm going to choose the ML rag which means I'm only going to go against the data in my database. So let's ask it what is the primary function of NASA. And one of the things you you want to do is you you want to go and test the different LLMs to see which one gives you the results that you're looking for. And so here it's come back with an answer primary function of NASA includes not only the exploration of space but also the continued responsibility and aeronautics.

Um let me do one more here. Let me just choose a a different LLM. So, we're going to ask when was the last space shuttle mission STS135. And again, it's going um to the NASA documents that I loaded. And here it comes back with a very succinct answer. And this is again why you want to test your LLMs because if I select a different LLM and submit the same question, it's going to give me a much longer answer than the earlier one here. So, the last space shuttle mission launched on July 8th, 2011 and landed on July 21st. So now let's do one more real quick. Let's go back and we're going to use MLG generate and we're going to ask it again for a recipe for fried chicken. And this is going to go against the the data the training data that's stored in the LLM because obviously I don't have any data about fried chicken in the NASA documents. And for some reason this does take a little bit longer. Um, but eventually it will give us a recipe for fried chicken. There we go. And there is the response here. So, uh, still looks pretty good there. I might have to give that one a try.

So, that is really it for my demo and presentation. I will go over to the chat and to see if we have any questions.