Since OpenAI released ChatGPT I along with the rest of the world have been captivated by what Generative AI can do. Things are moving so fast it's hard to keep on top of what you should be engaging with. What tools will we all be using in the near future and what skills do we need? I wanted to start tracking what I'm playing with and what tools I'm using. This is very incomplete but I'm going to try to recall what I've played with. --- ### 2025 I wanted to play with local LLMs even if they were under-powered on my personal M4 MacBook Air. So I installed https://ollama.com, downloaded [DeepSeek Coder](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct) and played with it on the command line. #### Grocery List Categorizer I created a little toy app to do something mildly interesting to see how a small LLM would do. Not perfect but it was fun to do. ``` code python import ollama client = ollama.Client() items = [ "Milk", "Steak", "Chicken Nuggets", # ... "Pinot Noir", "Malbec", "Jack Daniels" ] grocery_store_sections = [ "Bakery", "Beverages", "Breakfast", # ... "Snacks", "Seasonings", "Alcohol" ] def main(): for item in items: content = f"You are a helpful shopping assistant.\n" content += f"Here are the grocery store sections: {', '.join(grocery_store_sections)}.\n" content += f"Can you tell me what grocery store section I can find {item} in?\n" content += f"If you're not sure what section the item is in, just say 'UNK'.\n" content += f"You must only reply with a 1 or 2 word response for the name of the grocery store section name.\n" #print(content) response = client.generate( model="deepseek-coder-v2:lite", prompt=content, options={ "temperature": 0.2, "top_k": 10 }) print(f"{item.ljust(20)}: {response.response}") if __name__ == "__main__": main() ``` Output looks like this: ``` (shoplist) ~/code/shoplist ❱ /Users/demian/code/shoplist/.venv/bin/python /Users/demian/cod e/shoplist/main.py Milk : Dairy Steak : Meat & Seafood Chicken Nuggets : Frozen Foods ... Pinot Noir : Alcohol Malbec : UNK Jack Daniels : Alcohol ``` #### Debater I wanted to create another novel (for me) test to have the LLM debate with itself so I created this. The prompts setup debaters to talk back and forth. ``` code python import ollama from colorama import Fore, Back, Style, init import sys init(autoreset=True) model_name = "deepseek-coder-v2:lite" #temperature = 0.5 top_k = 5 client = ollama.Client() global_prompt = "You are an opinionated debater. " global_prompt+= "You should state a counter argument to the position presented to you. " global_prompt+= "You should not re-state the question you received. " global_prompt+= "You should reply in 3 sentences or less.\n " def generate_response(sender, prompt, i): temperature = 0 + (i * 0.05) if (temperature > 1): temperature = 1 if (temperature < 0): temperature = 0 top_k = 5 + (i * 5) if (top_k > 10): top_k = 10 if (top_k < 1): top_k = 1 stream = client.generate( model=model_name, prompt=prompt, stream=True, options={ "temperature": temperature, "top_k": top_k }) foreground = Fore.GREEN if (sender == "Vegan"): print(f"\n{Back.RED}Vegan{Style.RESET_ALL}") foreground = Fore.RED elif (sender == "Meat Eater"): foreground = Fore.BLUE print(f"\n{Back.BLUE}Meat Eater{Style.RESET_ALL}") response = "" for chunk in stream: chunk_response = chunk['response'] print(f"{foreground}{chunk_response}{Style.RESET_ALL}", end='', flush=True) print() return response def meat_eater(input, i): prompt = global_prompt + "You are a meat eater. " prompt += f"You must refute this specific statement with contrary arguments: {input}\n" return generate_response("Meat Eater", prompt, i) def vegan(input, i): prompt = global_prompt + "You are a vegan. " prompt += f"You must refute this specific statement with contrary arguments: {input}\n" return generate_response("Vegan", prompt, i) def main(): vegan_response = "Meat is murder why do you think it should be legal?" for i in range(1, 21): meat_eater_response = meat_eater(vegan_response, i) vegan_response = vegan(meat_eater_response, i); if __name__ == "__main__": main() ``` The output looks like this. The results are circular and dumb but amused me for a bit. ``` Meat Eater I don't believe that meat consumption equates to "meat is murder." The argument that meat eating is morally wrong overlooks the complexity of food production and cultural practices. Moreover, many cultures have historically relied on meat as a significant part of their diet for nutritional reasons. Legalizing something because it has negative connotations can be overly simplistic and disregards individual choices based on personal beliefs and values. Vegan As a vegan, I understand that everyone has the right to choose what they eat based on personal beliefs and values. However, it's important to consider the broader implications of dietary choices, such as animal welfare, environmental impact, and health concerns. While veganism can be beneficial for certain aspects of one's well-being, relying solely on plant-based diets may not provide all necessary nutrients for maintaining optimal health, especially in terms of protein, vitamin B12, and iron. Meat Eater As an informed individual, I recognize that personal dietary choices reflect one's values and ethical considerations. However, there is substantial evidence suggesting that reducing meat consumption can have significant health benefits, including lower cholesterol levels, reduced risk of heart disease, and a smaller environmental footprint due to less impact on deforestation and greenhouse gas emissions associated with livestock farming. These factors should be considered alongside personal preferences for a more balanced perspective. Vegan As a vegan, I understand that everyone has the right to choose what they eat based on personal beliefs and values. However, it's important to consider the broader implications of dietary choices, such as animal welfare, environmental impact, and health considerations. While veganism can be beneficial for certain aspects of well-being, relying solely on plant-based diets may not provide all necessary nutrients for everyone, especially when considering protein sources and long-term health outcomes. ``` ### 2024 At work we wanted to boost adoption. I really wish I could cite this and for all i know I have the details off but I heard a story about the British navy trying to fight scurvy. The sailors didn't like sauerkraut so what the leadership did is make it so that only the officers could have sauerkraut. The sailors thought they were missing out on something so they asked if they could have some. Eventually the leadership let the sailors have sauerkraut, the sailors at it up with enthusiasm and scurvy was no longer a problem. I didn't even know sauerkraut had vitamin C. So I required all my engineering leaders to do the take home exam we give to candidates. This was great in getting my leaders to understand that Generative AI is truly transformational and it also ginned up some FOMO on the part of our engineers. Engineers inevitably asked what the leaders were up to with an IDE open and leaders talked about what they were doing with their people. The leaders were generally from a technical background but there was a wide spectrum from leaders who haven't written code in anger for 10 years to leaders who still keep in good touch with the code. I made sure to go through the exercise myself. It took me 2 hours. My leaders took anywhere from 20 minutes to 2 hours. I asked that when they were done they demo their results to me and talk through how they did it. There was already some uptake in the our organization and this exercise didn't turn everyone into a true believer but it did give us a bump in adoption. ### 2023