How Andrej Karpathy uses LLMs

Andrej Karpathy, a former research scientist and a founding member of OpenAI, had a great video showing how he uses LLMs in his own life. We highly recommend everyone to watch this video, and try taking advantage of LLMs in daily life.

The following are what we find from the video particularly useful for hardware engineers:

Andrej uses LLMs for knowledge-based queries and internet search. He recommends starting a “New Chat” every time when switching topics, and keep the queries as concise as possible. This is because LLMs work on sequences of tokens, and they have limited working memories. Irrelevant topics within the same context window can distract LLMs, potentially decreasing the accuracy and model performance
Consider using “thinking models” for complex problem-solving. “Thinking models” are additionally trained with reinforcement learning to discover problem-solving strategies. These models emulate human-like thinking processes and often lead to higher accuracy in tasks requiring reasoning, such as math and coding. That also means, “thinking models” may take a long time to generate answers and conclusions. For OpenAI, any model with a name starting from “o”, e.g., “o1”, is a “thinking model”
Local files can be fed into LLMs’ context window, i.e., Retrieval Augmented Generation (RAG), and LLMs can then parse and summarize the documents for you
Certain LLMs have memory features that allow them to save information from conversation to conversation. For example, ChatGPT records a summary of what it learns about the user in a memory bank. This database of knowledge is then prepended to future conversations, allowing the LLMs to better understand the user over time. Users can also edit the memory banks to manage the information stored there