“Be formless, shapeless, like water. Now, you put water into a cup, it becomes the cup. Put it into a teapot, it becomes the teapot. Now, water can flow, or creep, or drip, or crash. Be water, my friend.” – Bruce Lee
When martial artist Bruce Lee offered the above advice about being adaptable, I doubt that he was referring to mathematical optimization. But these words of wisdom are certainly relevant to optimization algorithms.
Searching a design space is a lot like navigating a mountain range, and we know that no two mountain ranges are alike.
Even while traveling within a given range, you are likely to encounter several different types of terrain – smooth and rolling in some areas, rocky and rugged in other areas. Many design spaces are like this, as well.
Yet most optimization search algorithms use a fixed strategy for every problem, even though it is often impossible to predict the characteristics of a newly defined design space. Continue reading