project and event reflection9mo at archive captureWritten by Puviin Varman
One of the best ways to understand something is to try to break it
A practical reflection on learning LLM security through prompt-injection and capture-the-flag challenges.
Only a small 160 × 84 source thumbnail was retained. It is shown at its archival size and is not used as a hero image.
Original LinkedIn post text
One of the best ways to understand something is to try to break it
.
.
.
While scrolling on Luma, I came across an event by AI Tinkerers. This was at the time AI was really blowing up. The event? A capture-the-flag challenge. Honestly, I didn’t even know what that meant at the time.
The task? Crack an LLM to reveal a secret code. Each stage of the game got harder as the LLM got smarter with every attempt and round. It gets progressively harder until it was impossible to break.
I went prepared, of course I tried a similar test at home (hehe) and got through…but surprise surprise during the live event I was unable to crack the LLM.
Lesson learnt:
- LLMs get smarter over time - yes, were going to need terminator soon :-) --
- yes, LLMs can be hacked (read: prompt injection)
Here’s what I realised:
- To win, you had to curate your solution exactly to the problem.
- If you used an overkill solution too early, it wouldn’t work later.
- As the AI learns your tricks incrementally, it will eventually block you completely.
Feel free to try it out at https://hackmerlin.io/. Let me know if you get through :-)
On my next post I will share about the hackathon I joined a few weeks later.
That ended a bit differently - and the solution we built can actually be useful in a wide range of real projects. Hopefully guiding you to rabbit holes of your own.
#2025IsTheBestYearToBuild 🚀
