“Be formless, shapeless, like water. Now, you put water into a cup, it becomes the cup. Put it into a teapot, it becomes the teapot. Now, water can flow, or creep, or drip, or crash. Be water, my friend.” – Bruce Lee
When martial artist Bruce Lee offered the above advice about being adaptable, I doubt that he was referring to mathematical optimization. But these words of wisdom are certainly relevant to optimization algorithms.
Searching a design space is a lot like navigating a mountain range, and we know that no two mountain ranges are alike.
Even while traveling within a given range, you are likely to encounter several different types of terrain – smooth and rolling in some areas, rocky and rugged in other areas. Many design spaces are like this, as well.
Yet most optimization search algorithms use a fixed strategy for every problem, even though it is often impossible to predict the characteristics of a newly defined design space.
Selecting an optimization algorithm that uses a single strategy is like packing climbing tools and supplies for a single type of terrain without knowing what type of terrain you will actually encounter or how long the climb will take. Such poor preparation on the part of a mountain climber would seem foolish, but many optimization studies are based on equally bad planning.
A wise mountain climber prepares for the type of terrain he expects to encounter, but also carries a variety of tools in case he needs to adapt to other types of landscapes.
How do you plan properly when you don’t know what situation you are planning for? And how can an optimization algorithm effectively search a design space when its landscape characteristics are unknown?
The answer is very simple. “Be water, my friend.”