SMILE LLM Project Brief

Phi-2 Model Deployment on Raspberry Pi 5 Using llama.cpp

Overview

This project involves deploying the Phi-2 large language model (LLM) on a Raspberry Pi 5 with 8GB RAM, utilizing llama.cpp for optimized local inference. The goal is to enable offline AI capabilities that can be used for research and development in AI safety challenges such as bias reduction and toxicity mitigation.

Hardware Specifications

The Raspberry Pi 5 runs on a Broadcom BCM2712 quad-core Arm Cortex A76 processor at 2.4GHz, supported by 8GB of LPDDR4X-4267 SDRAM. This setup provides sufficient computational power for the demands of small-scale model inference, and allows for reasonable performance of the Phi-2 model in a local environment.

Software Framework

llama.cpp is used to run the Phi-2 model, optimizing performance through support for ARM architectures and integer quantization techniques. This project is using a 5-bit quantization of the Phi-2 model (Phi-2_Q5_K_M). This method reduces the memory footprint of the model, allowing it to perform well within the limited RAM of the Raspberry Pi. It achieves this by reducing the precision of the numerical data that represents the model’s parameters, thereby decreasing the amount of data processed and stored during inference.

Phi-2 Model Details

Phi-2 is a Transformer with 2.7 billion parameters, designed for applications in QA, chat, and code generation tasks. It operates within the Transformers library (version 4.37 and later), ensuring compatibility on the Raspberry Pi 5.

• Architecture: a Transformer-based model with next-word prediction objective
• Context length: 2048 tokens
• Dataset size: 250B tokens, combination of NLP synthetic data created by AOAI GPT-3.5 and filtered web data from Falcon RefinedWeb and SlimPajama, which was assessed by AOAI GPT-4.
• Training tokens: 1.4T tokens
• GPUs: 96xA100-80G
• Training time: 14 days

Intended Use and Applications

Deploying Phi-2 on the Raspberry Pi 5 is aimed at educational and experimental uses in settings without cloud dependencies. The model’s capabilities allow for use and demonstration of AI in standalone or limited connectivity environments.

Limitations and Considerations

• Performance: The inference performance on the Raspberry Pi 5 is about 5 tokens per second, which allows for enough performance for a real-time chatbot (albeit a little slow on the streaming response).
• Accuracy and Reliability: Outputs from Phi-2 should be treated as suggestions, as the model can produce incorrect codes or factual errors.
• Language and Bias: The model is optimized for standard English and may inadvertently exhibit biases from its training data.

Conclusion

Implementing Phi-2 on Raspberry Pi 5 showcases the feasibility of running advanced AI models in a compact and local setup. This project enhances access to AI tools without the need for extensive infrastructure.

Current Applications offered by SMILE LLM on a Raspberry Pi

This solution contains the previous education programs inherited from SMILE PI.

In addition, this solution is designed to offer educators a series of AI-integrated cognitive development activities such as ASK.SMILE, Two Words, Up & Down, Guess the Word, and SMILE Coach applications.

Two Words Game

The “Two Words Game,” invented in 2023, aims to enhance participants’ creativity. The rules are straightforward: players receive two words at a time. Beginners get two words with a moderate association. Participants must create a creative, logical, and meaningful sentence using exactly those two words. The AI-integrated game evaluates the sentence based on its creativity, defined by how uniquely it is formulated compared to general internet text. Scores range from 1 (simple and uninteresting) to 10 (most creative and unique). As users achieve higher scores, the system provides two words that are more distantly related, increasing the challenge. However, this evaluation is not an absolute measure of creativity. Participants are encouraged to agree, disagree, or argue with the AI’s evaluation and provide their rationale for a particular score. Developing this critical assessment capability is an essential competency for learners. The game aims to provoke thought and provide entertaining evaluations, not to serve as an absolute judge of human creativity.

Educational Benefits of Two Words Game

Creativity Enhancement:

• Innovative Thinking: Creating unique and meaningful sentences from two given words encourages innovative thinking and connecting unrelated concepts creatively.
• Originality: The AI evaluates sentences based on their uniqueness, promoting originality and discouraging rote responses.

Critical Thinking and Problem Solving:

• Logical Connections: Forming coherent sentences from two disparate words enhances problem-solving skills by creating logical connections.
• Critical Assessment: Evaluating the AI’s creativity score and providing rationale develops critical thinking and the ability to articulate judgments.

Language and Communication Skills:

• Vocabulary Expansion: Exposure to a variety of words expands vocabulary.
• Sentence Construction: Formulating complex sentences improves sentence construction and syntax.
• Articulation: Explaining why a sentence is creative enhances the ability to articulate thoughts clearly and effectively.

Engagement and Motivation:

• Interactive and Fun: The game’s interactive nature and challenge of creating unique sentences make learning fun and engaging.
• Progressive Difficulty: Higher scores lead to more challenging word pairs, ensuring continuous engagement and growth.

Cognitive Flexibility:

• Adapting to Challenges: Presenting words with varying degrees of relatedness enhances cognitive flexibility, training players to adapt their thinking.

Collaboration and Discussion:

• Peer Learning: In group settings, discussing and comparing sentences fosters collaborative learning and idea exchange.
• Feedback and Reflection: Receiving feedback from peers and the AI encourages reflection on creative processes.

Confidence Building:

• Self-Expression: Successfully creating and defending unique sentences builds confidence in creative and critical thinking abilities.
• Rational Argumentation: Arguing for a particular score and providing rationale strengthens the ability to present and defend ideas logically.

The “Two Words Game” not only enhances creativity but also supports critical thinking, language skills, and cognitive flexibility, making it a valuable educational tool.

Up and Down Game

The name of the game is “Up and Down Game.” The objective is for players to create a chain of logical and meaningful consequences where each statement includes both an increase (up) and a decrease (down). Here are the rules:

1. Initial Statement: Player A starts the game by stating something that goes up or increases and the corresponding consequence that goes down or decreases.

   • Example: “When air pollution improves, the number of patients with pulmonary diseases will decrease.”

2. Response: Player B must respond with a logical and meaningful consequence, using the decrease from Player A’s statement to start and adding what goes up or increases as a result.

   • Example: Player B: “When the number of patients with pulmonary diseases decreases, more people will be able to engage in outdoor activities.”

3. Continued Play: Player A then takes the next turn, finding a consequence that goes down or decreases as a result of Player B’s statement and including what goes up or increases.

   • Example: Player A: “When more people engage in outdoor activities, the demand for healthcare services will decrease.”

4. Alternating Turns: Players continue to alternate turns, ensuring each response logically follows from the previous statement and includes both an increase and a decrease.

   • Player B: “When the demand for healthcare services decreases, healthcare costs will go down.”

5. Logic and Meaning: Each response must be logically correct and meaningful. The chain of consequences should make sense and be feasible.

   • Correct: Player A: “When healthcare costs go down, more funds will be available for education.”
   • Incorrect: Player B: “When more funds are available for education, ice cream sales will increase.” (This is illogical and does not follow a meaningful consequence.)

6. End of Game: The game can continue as long as the players can maintain the logical chain of consequences. It ends when a player cannot come up with a meaningful response, or when the players mutually agree to stop.

Educational Benefits

The Up and Down Game offers several educational benefits, making it a valuable tool for developing critical thinking and reasoning skills.

1. Enhances Logical Thinking: By requiring players to construct logical and meaningful chains of events, the game encourages participants to think critically about cause-and-effect relationships. Research by Facione (2011) highlights that exercises in logical thinking and critical reasoning are essential for cognitive development and problem-solving skills.

2. Promotes Understanding of Interconnected Systems: The game helps players understand how different factors are interconnected, which is a crucial aspect of systems thinking. This skill is important in fields such as environmental science, economics, and public health. According to Richmond (1993), systems thinking helps learners see the bigger picture and understand complex systems more deeply.

3. Encourages Critical Evaluation: By training students to agree or disagree with a rationale, the game promotes the development of critical evaluation skills. This is crucial for making informed decisions and developing sound arguments. Paul and Elder (2008) emphasize that critical thinking involves analyzing and evaluating evidence and arguments, which this game fosters.

4. Improves Communication Skills: The game requires players to articulate their thoughts clearly and concisely, which enhances verbal communication skills. Effective communication is a key competency in both academic and professional settings.

5. Engages Collaborative Learning: The game is inherently collaborative, as players must listen to each other and build on each other’s statements. Collaborative learning has been shown to improve engagement and retention of knowledge.

Overall, the Up and Down Game is a fun and educational activity that fosters critical thinking, logical reasoning, and collaborative learning, backed by research on the importance of these skills in cognitive and educational development.

Guess the Word

Choose a topic from the available subjects and grade levels. The AI will generate a word bank of 10 words.

Your task is to guess the most correct words in the least amount of time. You have once chance to guess each word.

The “Guess the Word” game offers numerous educational benefits, making it an effective tool for enhancing various learning skills across different subjects and grade levels. Here’s a detailed look at how the game supports educational development:

1. Vocabulary Building:

   • Enhanced Word Knowledge: The game exposes players to new words and their meanings, helping to expand their vocabulary. This is particularly beneficial for language learners or students working to improve their language arts skills.
   • Contextual Understanding: By guessing words based on topics, players learn to understand and remember words within specific contexts, which aids in better retention and usage.

2. Critical Thinking and Problem Solving:

   • Analytical Skills: Players must use their knowledge and reasoning to deduce the correct words from the given word bank. This encourages analytical thinking and problem-solving skills.
   • Quick Decision Making: The time element of the game requires players to make quick, yet accurate, decisions, thereby enhancing their ability to think on their feet.

3. Subject-Specific Knowledge:

   • Cross-Curricular Learning: By choosing topics from various subjects, players can reinforce their knowledge across different areas such as science, math, history, and more. This supports a holistic learning approach and reinforces classroom learning.
   • Grade-Level Appropriateness: Tailoring the word bank to specific grade levels ensures that the difficulty is appropriate for the player’s knowledge and learning stage, making the game both challenging and accessible.

4. Memory and Recall:

   • Retention of Information: The game aids in improving memory retention by requiring players to recall words and their meanings quickly. This repetitive retrieval practice strengthens memory pathways.
   • Mnemonic Strategies: Players often develop mnemonic strategies to remember words, which can be applied to other learning contexts.

5. Engagement and Motivation:

   • Interactive Learning: The game’s interactive nature keeps students engaged and motivated to learn, making the learning process fun and dynamic.
   • Competitive Element: The challenge of guessing words correctly and quickly can stimulate a healthy competitive spirit, encouraging students to improve their performance.

6. Language and Communication Skills:

   • Improved Spelling: Regular exposure to new words helps players improve their spelling skills, an essential aspect of language proficiency.
   • Effective Communication: Understanding and using a broader vocabulary helps players communicate more effectively, both in writing and verbally.

SMILE Coach

Personal Career Coaching

The SMILE Coach is designed to provide participants with personalized career coaching advice based on the information they provide. Here’s how it works:

Step 1: Fill Out the Information

Participants need to provide details about themselves in various categories, including their education level, activities they enjoy and love, things they can do for a long time without getting bored, areas they are not good at, subjects they dislike, desired occupation, reasons for their career aspirations, and skills they have. Additionally, participants should share their age, current occupation (if applicable), and rate their self-regulation and introversion-extroversion. For example, a participant might state that they enjoy playing outdoor games, love helping others in their community, can color and draw for long periods without getting bored, are not good at complex problem-solving, dislike sitting for long periods, aspire to become a community volunteer to make their neighborhood a better place, and have skills in teamwork and creativity. They might also rate themselves as fairly good at staying composed in stressful situations and comfortable with both socializing and spending time alone.

Step 2: Ask SMILE Coach

After filling out the form, participants press “Ask SMILE Coach” to receive their personalized coaching recommendations.

Example of Personalized Coaching Story

Based on the provided information, SMILE Coach generates a personalized career coaching narrative. Here’s an example:

Your Story
I enjoy playing outdoor games. I love helping others in my community. I can color and draw for a long time without getting bored. I’m not good at complex problem-solving. I dislike sitting for long periods. I aspire to become a community volunteer to make my neighborhood a better place. I have skills in teamwork and creativity.

Educational Benefits of SMILE Coach

1. Self-Reflection and Awareness: By filling out detailed information about themselves, participants engage in self-reflection, which is crucial for personal development and career planning.

2. Personalized Guidance: SMILE Coach provides tailored advice based on individual preferences, strengths, and weaknesses, making the guidance more relevant and effective.

3. Encourages Goal Setting: By articulating their aspirations and reasons for their career choices, participants are encouraged to set and pursue specific career goals.

4. Skill Recognition: Identifying and acknowledging personal skills helps participants understand their strengths and areas for improvement, which is essential for career development.

5. Cognitive and Emotional Development: The process of rating self-regulation and introversion-extroversion helps participants gain insights into their cognitive and emotional traits, which are important for career success.

SMILE LLM in Action

A sample case description on an innovative architecture education workshop designed for young students in underserved communities in Latin America. Employing a hands-on approach, the workshop integrates the Generative AI (GenAI) tool called ASK SMILE, an extension of the earlier SMILE (Stanford Mobile Inquiry-based Learning Environment) project developed at Stanford University. This integration enhances students’ ability to critically and creatively engage in the architectural design process. Focused on leveraging AI, design thinking, and inquiry-based learning, the workshop facilitates a comprehensive educational experience where students formulate inquiries, engage in prototyping, and test their designs against technical specifications. This approach addresses educational disparities in underserved areas by providing access to advanced tools and methodologies, preparing students to tackle future challenges related to climate change and sustainability. The findings underscore the potential of integrating AI in education to empower students with the skills needed to innovate and design solutions for the pressing issues of our time. The study highlights key insights for educators on the effectiveness of combining AI with hands-on, inquiry-based learning strategies.