How I Create Self-Learning and Self-Documenting Skills in Claude Code

I build Claude Code skills that teach themselves and write their own documentation, getting better every time I use them.

Alex Hillman
Written by Alex Hillman
Collaboratively edited with JFDIBot
JFDI

Most people write a skill once and it stays static forever. I wanted skills that improve themselves.

I walk through how I build Claude Code skills that learn from usage, capturing edge cases, updating their own documentation, and getting better every time they run. The mechanism is structured feedback loops built directly into the skill workflow.

The result: my skills started simple and gradually became comprehensive, without me having to manually maintain them.

Full Transcript

So I wanted to share a quick example of how I will wrap up a session and turn it into a set of Claude Code skills. In this case it’s also creating some documentation, which is useful and I think a good practice here as well.

So I’m doing some work on the part of my system that allows me to interact with Claude Code through Discord. Without getting into the weeds of how it works, think of it as a tool that allows Claude Code to post to Discord, basically do anything that a user would do, and in many ways more, because Discord also has some features that users don’t get to use, like interactive buttons and stuff like that. Messages, threads, buttons - all those things are kind of like building blocks for Claude.

What I wanted to do is take what we built for very specific workflows and say, “Hey, what you actually built is a bunch of little building blocks.” We built them for one thing in mind, but let’s make those building blocks more useful for later. And that’s a really good way of thinking about this. Build things for a specific purpose and then ask Claude, “Is there anything more universally useful here? Can we turn it into building blocks?” The jargon you might hear programmers use is “primitives.” I like to think of them as Legos, or the way IKEA uses the same hardware to connect different pieces of laminated fiber board. There’s like an IKEA element to this.

So I asked it, “Hey, can you review everything we’ve done in the session? Suggest documentation improvements, skills, and what we should make.” And it made some documentation recommendations. If you aren’t technical, some of this might look foreign, and to a degree that’s okay, because I can have it translate to a level of understanding that works for me. You can have it translate to a level of understanding that works for you. When I don’t understand something, I just ask it to explain it as best as it can. Or sometimes I’ll even give it a point of reference and I’ll say, “Here’s what I do understand. What am I missing?”

So that’s the documentation piece, but then it also recommends some skills. And I said, “Cool, do the docs and then let’s talk about the Discord bridge skill, that seems useful.” And so it went ahead and just did the docs. Docs are done. I’ll go read those later. The skill is really what’s at stake here.

So remember I said we created these very specific workflows and I asked it to say, “What are the useful building blocks that we could make more universal?” And it goes, “Actually there’s a bunch here. You could send messages programmatically, you could post formatted content, you could do interactive stuff, you can manage threads” - again, basically all the stuff that a user would do, and then some.

And then it asks the question: is this a skill that Claude is going to use when I ask it to do Discord things, or is it documentation for when you’re building scripts? Is this reference material, or is this the thing that you actually want the assistant to be able to use? For me, the answer is I want the assistant to actually be able to use this. This is not simply documentation. This is that actual building block functionality.

And so in this case I said, “I think I get the question, so good question. Help me think this through.” And I kind of explained that the Discord bridge is currently tied into the thing that we just built, and I want those primitives, maybe a mix of primitives and some usage examples. The reason I always look for usage examples is because Claude Code loves to know what success looks like. If we show it a couple of examples that already work, it can say, “Oh, you want something like this but in this other way.” When it invokes itself, it can look at those examples and it’s not looking at a blank page, the same way that it’s easier for us to come up with new ideas when we see existing ones. Same thing for Claude Code.

So it broke it down and said, “Here’s what we could do. We’ll do a send message primitive, send file, send embed, send with buttons, the typing indicator, some usage patterns that it would enable, and what the skill file would contain.” And I’m reading through all this and going, “Yeah, that makes sense.” I specifically said, “Hell yeah, now we’re talking.”

If it didn’t make sense, I could ask more questions, say, “Yeah, like this, but not like that” or “let’s start with these.” Depending on how well I understand the system or how big and robust it is, I might start there, but more commonly I’m like, let’s just do one thing. So in this case I said, “Before we do this, can you sketch out a workflow diagram, just for easier reading, about how the bridge works?” I have it do this often, especially when we’re building skills around interconnected parts. In this case it’s Claude Code and Discord.

So let’s open this up and see what we’ve got. These are just little throwaway text areas. It put together this little overview, and again, for the business owners, you don’t necessarily need this. I think this is useful for education. And again, if this doesn’t make sense to you, you could say “explain this more deeply” - or “create a story scenario” is a great one. I love using Claude Code to have it explain things in terms that make sense to me. So if there are things that make sense to you, tell it: “I’m a business owner. Explain this to me like I’m a business owner, not a programmer.” It can do that, and if things don’t make sense, you can always refine it.

I actually haven’t looked at this before I pressed record, but everything in here makes sense. Okay, I’m going to say that’s good. And I say, “This looks great, and I love the examples you suggested for the skill. Go ahead and build it.” And then what I would generally say is, “Use plan mode and ask me any clarifying questions that you might have.”

So what it’s going to do is read through all that stuff, all of our documentation, and it is going to create a new skill folder that’s got the skill file in it. That is your markdown file that has the instructions. And then it’s also going to create all the little bits of code that it needs in order to poke the Discord API the way that we want it to, consistently. This will probably run for the next four or five minutes, maybe a little bit longer, that’s my hunch. And at the end, I’ll have a skill.

The cool thing is that means I can either explicitly mention that skill when I want Claude Code to do things in Discord, whether that’s inside a manual thing that I’m doing or inside my automated workflows that run on scheduled jobs and things like that. So I hope this is a concrete example that makes sense. It is a little more on the technical side of things, and like I said, I want to give people examples that are not that. So for the business folks specifically - if there’s anything here where you’re like, “You lost me completely,” please ask. I want to try and break it down, and we’ll see if maybe Claude can help us do that too.

← All posts