There are so many methods that you can use when you first learn something — break things apart, mimic others, reason by metaphor, ask a coach, etc. You start to see yourself make progress — and fast. As long as you try, even moderately, you see improvement. You start gathering more and more building blocks and begin to even excel at each one of these. It’s almost too easy.
Then suddenly you hit a wall. Your progress stagnates. All your newly acquired tactics seem to have maxed out. Welcome to the intermediate zone. This is where you have to grind, where you’ll have to experiment, where you’ll have to fail and take risks in order to jump out of your local maxima. Progress will feel non-existent. You have to “trust the process”. This stage is the hardest part of learning any skill well.
Why is it so hard? What is different between the beginner stage and the intermediate stage?
The beginner stage is all about the pieces. You learn everything in discrete units — especially if you learn from someone else. The more effective the teacher, the more they optimized the beginning stage for you. You’re learning how to efficiently dice an onion, how to have a good tennis swing, and recite vocabulary in French. Your environment is static; you don’t need to respond to it.
The intermediate stage is all about the integration. It’s not enough to dice an onion, now you need to make the dish; it’s not enough to swing at a ball, now you need to win the point; it’s not enough to know vocabulary, now you need to persuade someone. Integration is more than the sum of its constituent parts. How you put them together and use them in one integrated motion is everything. Your environment is dynamic; you have to respond to it.
All the tactics you had at the beginner stage won’t work anymore. You have to start developing principles that you hone over continuous, diverse practice. You need to try different angles, invert the problem, reduce the dimensionality, abstract it, and break it apart again into new units1. Someone can guide you through a particular case or offer you hints, but no one can expressly tell you what to do in every single scenario. You’ll need to keep sorting between principles and practice, over and over again. Some days you’ll feel like you’re making progress; other days you’ll feel like you’re literally regressing. But over a long enough exposure horizon with enough diversity of exploration, you’ll see that you are improving.
Eventually, you’ll hit a point where you realize that you’re actively pushing the boundaries of what you’re knowing to be possible. You’re trying to break every single rule you’ve developed in this skill, redefining your own methods, re-constructing your own principles to become more and more fundamental. You’re seeking razor thin optimization because you need the edge. When this happens, you’re done with the intermediate zone.
Claude Shannon’s excellent essay on Creative Thinking is worth a read for more on problem solving. I believe this applies to any domain.
From my observation, a lot of analytically strong people suck at synthesis. And add to that, they are not even self aware about this. True mastery requires synthesis or what you call integration of different ideas/concepts in a domain.
It is easier to illustrate this using Chess. The game has 3 distinct stages - the opening, the middle game and the end game. Opening and end game stages are mostly technique and uses the analytical part of the brain. The middle game, however, with the pieces all over the board requires synthesis. Often, it is not even clear what one should do.