Feature #3607
Implementing AI into Hope project
| Status: | Assigned | Start date: | 03/04/2024 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | 04/30/2024 | |
| Assignee: | Kasthuri G R | % Done: | 0% | |
| Category: | - | Spent time: | - | |
| Target version: | - |
Description
Take input words from Hope, generate sentences using large language models (LLMs) with basic commonly used words
History
#1 Updated by Kasthuri G R about 2 months ago
Google has released an open AI model Gemma(https://blog.google/technology/developers/gemma-open-models/). I am able to generate sentences for the given input words.
I have provided the GSheet link below with the input words and corresponding AI results generated by Gemma model
[[https://docs.google.com/spreadsheets/d/1CluoU2D38b6nuRMt4AJDTpU6_GFoB0e3VcQOo0_QJ0Y/edit?usp=sharing]]
This is open source model and it gives expected results. We will have to design the prompts to get better results based on the user needs. I will set a deadline based on the list of features we deciding
#2 Updated by Kasthuri G R about 2 months ago
To get inputs from special educators, made a sample UI in python that user can upload image and it will generate sentences for the given image. I have implemented the functionality. Yet to make the UI better with background color and alignment.
#3 Updated by Kasthuri G R about 1 month ago
Made a sample UI that allows to give input image and generate sentence. The model output varies like below. I am trying to figure out the method to filter the output.
For eg: Given Prompt "construct 5 basic proper sentences about {input word} and its habits for primary school students"
AI output
Step 1/5
1. Parrots are beautiful birds that come in many different colors and sizes.
Step 2/5
2. Parrots are very smart and can learn many tricks and even talk.
Step 3/5
3. Parrots are social birds and love to be around other parrots and people.
Step 4/5
4. Parrots need a lot of attention and exercise to stay healthy.
#4 Updated by Anil Prabhakar about 1 month ago
- Tracker changed from Bug to Feature
- Project changed from CREATE to AAC
- Due date set to 04/30/2024
- Status changed from New to Assigned
There should be a Due Date for every task and an updated %Done. Cannot work without any deadlines.
- Do we have a creative common license to use these images?
- How are you imagining the use case, is this set up by a teacher, and the child clicks on a button, or is the child supposed to upload an image?
- This is not picture to speech (think AI into HOPE...as the subject of this Issue). I am expecting a communication tool, where a user selects 1 or more images from the KAVI-PTS library, AI generates a few possible sentences, and the child picks one of them.
#5 Updated by Kasthuri G R about 1 month ago
- File ThankYou.png added
- File Classroom.png added
- File Nice_to_meet.png added
- File Sorry.png added
Do we have a creative common license to use these images?
I will use the KAVI Library images
How are you imagining the use case, is this set up by a teacher, and the child clicks on a button, or is the child supposed to upload an image?
This is a sample UI we decided to create and show it to special educators in Mira school to get their ideas on the requirement. So we can plan the development in both Hope and Kavi-PTS based on their feedback
The sample use cases I have considered are below
1. AI to generate Sentences with the "Thank You, Sorry, Nice to meet you" which children can use. These are the pictures under Quick talk in the Kavi Library
2. Common sentences used by the children in Classroom. Class room is also the picture from our image Library
The respective AI responses for the above inputs are uploaded.
#6 Updated by Anil Prabhakar about 1 month ago
- Use the > to quote, otherwise your reply is mixed with my comment.
- Sentences are quite ambiguous and will become frustrating. We may need more than one image to create a sentence. Let us try with 2 images.