Dayward AIWeChat Mini Program
From Frontend Engineer to Agent Engineer in 30 Days
Week 2 ยท D12๐Ÿ”’ LockedAbout 6 hours

Long-Term Memory: pgvector, Embeddings, Chunking, the memory_search Tool

Give the agent long-term memory with pgvector: chunk historical content, generate and store embeddings, then wrap it into a memory_search tool the model can call.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Create a table in Postgres with pgvector to store embeddings and run a similarity query
  2. Implement a write pipeline that chunks long text and generates embeddings
  3. Wrap similarity search into a memory_search tool and wire it into the agent built on days 5-6

D11 connected the chain back to the user: the worker generates, the gateway pushes in seq order, and a conversation finally makes a complete round trip. But this agent remembers only the current session โ€” not one word of what the user said last month. Today we fit it with cross-session long-term memory.

Plain-Language Walkthrough

Not recalling last month is not the same as not fitting in this round

D6 taught a move you may well misapply today: meeting minutes. Keep the decisions and throw the transcript away. That was a trade-off within one meeting, solving "this round will not fit." Today's problem is different โ€” last month's minutes were archived long ago and you cannot recall what was decided.

The two mechanisms point in opposite directions. D6's compression subtracts while assembling a request: the history is too long, so cut a stretch and replace it with a summary. Today adds: while assembling a request, fish a few relevant old facts out of a store that lives outside the session. The failure modes invert too: failed compression hits the context ceiling and the request errors outright; failed retrieval merely means "did not remember," and the conversation continues. Neither the same problem nor the same mechanism, and not one line of D6's thresholds and summarization carries over.

So how should that outside-the-session store be queried?

There are two ways to find a book in a library. One is the title catalog: you must know the title, and one wrong word finds nothing. The other is the subject shelf: everything in this row is about the same thing, and differing titles do not matter.

Keyword search is the title catalog. The user said three months ago "I am allergic to peanuts" and today asks "what should I avoid eating" โ€” those two sentences share no keyword, and LIKE '%avoid%' returns nothing. And that is memory's most common usage: people do not ask using the words they originally said.

Vector search is the subject shelf. It places each stretch of text into a coordinate system whose position is determined by what it is about; "peanut allergy" and "food to avoid" land next to each other, and sorting by distance brings it back.

So why not simply stuff every old fact into the context and skip retrieval? Do the arithmetic. An active user accumulates 200 memories over six months at about 400 characters each, and by D6's conservative one-character-per-token convention, all of it is 80,000 tokens. At a chat model's input price of 0.15 dollars per million tokens, every single round pays an extra 80000 / 1000000 * 0.15 = 0.012 dollars; at 20 rounds a day, one user costs 0.24 dollars a day. Retrieving only the 5 most relevant is 2,000 tokens, or 0.0003 dollars a round โ€” a factor of 40.

Money is not even the main point. Of those 200, perhaps 1 is genuinely relevant and the other 199 are noise, and the model gets pulled off course by them, dredging up "they returned something last month" to answer a question with nothing to do with returns. Every irrelevant item you put into the context lowers the model's hit rate. D6's closing line about retrieving only three to five was about exactly this.

The question therefore becomes very concrete: how does a machine compute "these are about the same thing"?

Sign in and link WeChat to read

Sign in, then link your WeChat mini program account on the account page. Linked accounts share one free article per day. Unlock additional articles with rewarded videos in the mini program.

Mini program QR code

Scan in WeChat to open Dayward AI. Open the same course and chapter, choose the rewarded video, then refresh this page after completion.

Interview questions

  • Why does an agent need a separate long-term memory instead of stuffing all history into the context window?ไธบไป€ไนˆ Agent ้œ€่ฆ้ขๅค–็š„้•ฟๆœŸ่ฎฐๅฟ†๏ผŒ่€Œไธๆ˜ฏๆŠŠๅކๅฒๅ…จ้ƒจๅกž่ฟ›ไธŠไธ‹ๆ–‡๏ผŸ
    Common in ChinaCommon overseasBasic#long-term-memory#rag#cost

    ๐Ÿ”’ Unlock this chapter to see the answer keys.

  • How does pgvector compare with a dedicated vector database, and how would you choose?pgvector ๅ’Œไธ“็”จๅ‘้‡ๆ•ฐๆฎๅบ“็›ธๆฏ”๏ผŒไผ˜ๅŠฃๅˆ†ๅˆซๆ˜ฏไป€ไนˆ๏ผŸไฝ ไผšๆ€Žไนˆ้€‰๏ผŸ
    Common in ChinaCommon overseasIntermediate#vector-database#pgvector#architecture

    ๐Ÿ”’ Unlock this chapter to see the answer keys.

  • How does the chunking strategy affect retrieval quality, and how do you pick a chunk size?chunking ็š„ๅˆ‡ๅˆ†็ญ–็•ฅไผšๆ€Žไนˆๅฝฑๅ“ๆฃ€็ดขๆ•ˆๆžœ๏ผŸๅˆ‡ๅคšๅคงๅˆ้€‚๏ผŸ
    Common in ChinaCommon overseasIntermediate#chunking#rag#retrieval-quality

    ๐Ÿ”’ Unlock this chapter to see the answer keys.

  • What matters when designing the parameters of a retrieval tool such as memory_search?ๆŠŠ่ฎฐๅฟ†ๆฃ€็ดขๅŒ…่ฃ…ๆˆ memory_search ่ฟ™ๆ ท็š„ๅทฅๅ…ทๆ—ถ๏ผŒๅ‚ๆ•ฐ่ฎพ่ฎกไธŠ่ฆๆณจๆ„ไป€ไนˆ๏ผŸ
    Common in ChinaCommon overseasDeep dive#tool-design#security#long-term-memory

    ๐Ÿ”’ Unlock this chapter to see the answer keys.

Comments