Setup the Optiease Local Chrome AI

A small note before you start. This guide takes about 20 to 30 minutes to read and follow, not counting the download or wait time. You will need the Chrome browser on a computer device like a laptop running Windows 10 or 11, MacOS 13 or newer, or any system that works in a similar way.

If you missed our earlier post, Introducing Optiease Local Chrome AI, it is worth reading first: XXXXXXXXXXXXXXXXXX.

If you run into any setup issues or have questions about the software, reach out. You can click here for free support or email admin@optieaseai.com. Everything is free. Our aim is to keep this open, simple, and helpful so more disability and healthcare professionals can benefit from it.

Download the App File & Model

You can set up the web app using this link: https://github.com/jxspam/optiease-ai-local-chrome-chatbot. If you are new to GitHub, the easiest way is to download the ZIP file. If you know version control, you can use
gh repo clone jxspam/optiease-ai-local-chrome-chatbot). 

After you download the ZIP, open it and extract the files. You want a normal folder, not a zipped one. The folder will likely sit in your Downloads area at first. You can move it to any place on your computer where you prefer to keep your tools.

For the best experience, always use the newest version of the code when updates come out.

By opening the index.html file inside the folder with Chrome browser. This will bring you into the chat interface. At the beginning the model is not installed yet, so tap the gear icon in the top right corner and select “Download Gemini Nano Model”. You can check your model status anytime inside the same settings area.

The model installation is estimated to take 10 - 15 minutes, the model status will show from “Starting download”, “Downloading model: XX% complete” to “Download Completed”.
Don’t refresh your page or close your browser at the mean time as it stops the installation progress directly. Once the download is complete if you check your chrome://on-device-internals/ you will realise that it has the file path and the folder size displayed (remember to refresh & my screenshot could be different to yours, as I set up before), notice the folder size is increased to 4GB.

Search up chrome://flags/#prompt-api-for-gemini-nano , enable them and choose “relaunch” (restart your chrome browser). This is the way you would allow the app to access in calling your local LLM i.e. Google Gemma Nano Model.

Once the model finishes installing and the Prompt API is turned on, refresh the page. You can start chatting with your local AI right away by typing text or adding images. The first reply may take 5 to 10 seconds as it warms up. All your chats stay stored inside Chrome’s Indexed Database on your device. That means everything stays local. Congrats!

You can explore the custom settings to change how the model behaves. A Large Language Model is a type of transformer that reads your words, holds the context, and predicts the next piece of text step by step. The output can change each time because the model uses probability to pick the most fitting next word.

The temperature setting controls how random the text can be. A setting around 1.0 usually gives a good mix of clarity and creativity. Top K limits how many of the most likely next words the model can choose from, and most people set it between 20 and 50.

You can also set a limit on how long the model’s replies should be. Shorter replies save time and reduce the amount of data processed in later chats, even though it is not required. You can cap the storage the app uses in your browser as well, up to 100MB.

If you have any issue in interacting with your AI no matter if it is your first time or times after, be sure to check out the session below (Common guidance on fixing issues). 

Common guidance on fixing issues

The common guidance below can solve 95% of the issues when you encounter any issue in interacting with your AI:

1. At the moment, the app development can be unstable. As we try our best to solve the edge cases and device compatibility issues, logics could still not yet well handled, models can crash on your Google Chrome, be sure to constantly monitor chrome://on-device-internals   to make sure the model status is always optimal

2. If you have the model downloaded and the status seem to be all healthy in the chrome://on-device-internals , put chrome://restart (make sure you have done and saved all the works in the other tabs if you are using them), it will restart and reinitialise the model and refresh the app page (Ctrl + R/ Command + R), if it still doesn’t work, 

This instruction can be obtained on the warning page below. Generally, it will show up if the model is downloaded but the app encounters some restrictions e.g. crashes, API not enabled when calling the model.

Advanced Use Case: Setup the server 

In fact, if you are happy with the current interaction, text and images support, and storage saved on your chrome browser (which can be cleared soon), then you have done! Congrats again! You shall celebrate if you have no prior programming experience, this is probably your biggest step in the technical field at the moment!

But if you would like to have some more extensive features such as uploading any text based file type such as pdf, pptx, docx, xlsx… ; acquire YouTube transcripts by putting the YouTube link; save all conversation history and files uploaded in a specific folder path; also, we may consider putting lots more on here.

Setup file allows user to run a server on your local laptop to save the data on specific folder path, converting any file into the text version (like pdf, pptx, docx, xlsx…) with the support of the MarkDown package, it is noted that without the server running, the app should still be able to behave, but may encounter error when you upload file other than images type. For Mac users, click on setup.sh, while Windows users, click on setup.bat (Windows Batch File). If you are a developer and have a Python Integrated Development Environment (IDE), running python setup.py can be a good alternative.

For Windows Users, it usually shows:

It takes around 15 - 25 minutes to finish the running of the server if you firstly set up. The server is successfully running when it arrives at this stage:

Right now you can specify a folder path to save your conversation history, upload a pdf file, try uploading with a YouTube link, everything mentioned so far can be achieved without the internet connection except for YouTube features. Since this is a small model with a limited window context (aka limited words it could take), especially when dealing with with exceptional large file or text processed, the chat interface will prompt “

With server setup - 1: Upload a pdf file

Before running a server, if you upload a pdf file to the Optiease AI Chat, the AI can only recognise the file name, but not the content.

Right now, with the server running at the backend, your server will first listen the pdf file request from yout application and convert it into a text based version, the converted text will be brought to your AI together with your prompt.

With Server Setup - 2: Upload a youtube link for transcripts

You can also find a public youtube video and upload it onto your Optiease AI Chat, and ask your AI to summarise or extract certain insights for you.

With Server Setup - 3: Save your conversation in a specified folder

You can also find a public youtube video and upload it onto your Optiease AI Chat, and ask your AI to summarise or extract certain insights for you. Create a folder, copy it as path

Go to your Optiease Chat setting panel, choose “Select Folder”, and paste in the folder path that you would wish the conversation to save. Remember to delete the “” open and close quotes.

A new folder is just being created after one conversation kicks off

Do note that if the server is not available, the session conversation history will be stored at the browser database by default.

Here was the list of the app API calling history records made to the server.

It is suggested that you always open the index.html file and setup.XX files whenever you’d like to interact with your AI. 

Next Step

Have fun and play around with it!
We are really excited to see this in real world practice. And we know, the current setup process is super complex, and can potentially become a huge barrier for people to access the platforms, even though we have already tried our current best job to make sure every package is right in place with the most convenient setup process that we know. We will find a way to constantly refine the process to make sure it reaches its optimum. 

If you run into any setup issues or have questions about the software, reach out. You can click here for free support or email admin@optieaseai.com. Everything is free. Our aim is to keep this open, simple, and helpful so more healthcare professionals can benefit from it.