scipy.optimize.curve_fit taught me to believe in myself more.
I used to be a confident, optimistic, and a do-it-all sort of medium-skilled programmer. Then on a hot and humid late summer day of July 2017, I grew up. I was now cognizant of the ‘Machine Learning Revolution’. It was baffling initially but got more and more interesting as I got to know more about it. (Yes, I used 3 more(s) in a sentence; we all have our demons.)
My first instinct was to hop on to the AI/ML express. Alas, I was ‘only’ a budding materials engineer, an Indian materials engineer. Programming prowess seemed illusive beyond a point. No wrong impressions here; Indians have brilliant minds. Every person needs to be put in an environment that relays the need to invent, be creative, be minimalistic. It can be an environment of freedom or a restricted one; it doesn’t matter. It just has to be right. Most Indian engineering programs, except select elite ones, don’t put us in such conducive environments. In 2020, many Indian universities are yet to catch up with the computing revolution, let alone ML.
I could write codes to solve basic problems, and I got good grades in computer programming classes. That’s where my foray into programming ended. The rest of my classes involved rote learning, a lot of button-tapping on calculators, and good hand-writing. That was undergraduate. Enter 2017, graduate school — one of those select elite ones.
July 2017. Machine learning. Aren’t C and C++ the most popular languages? Python? There is a language named after a species of snake? Photon is a language too? Oh, you meant Fortran. So the second ‘r’ is silent? Numerical differentiation and integration are used casually? Aren’t they for high funda stuff alone? November 2020: computational materials engineering is such a bounteous field. There is room to develop tools based on the rich theoretical knowledge-base in this field. I am glad I got comfortable with Python.
The time between July 2017 and Nov 2020 was like touching extremely hot and cold objects at once. You don’t know what to feel. You lose direction. At times, my friends (also materials engineers) and I would reminisce about how we were top of the class in undergrad programming, and at other times, despair about how much the scene of programming had changed in such a short span of time. All the while alluding to machine learning. Later, we would witness the utter rise of job offers based on AI/ML skills at the cost of traditional engineering skills or knowledge. It seemed as though all engineering branches were becoming more like freelance education and only computer science was to thrive professionally. We were obviously wrong, but I guess ‘mob mentality’ works in promoting scepticism as much as it does in aggression.
We liked programming. We valued it. We could assess its importance, growing importance. But the discomfort of not being good enough, not having the time to balance between learning traditional engineering topics and learning programming at current standards, not knowing how to harness the power of GPUs, and so on would ‘ebb and rise’. Mostly ebb.
Eventually, I broke out of my undergrad habits and learned to think of traditional engineering and programming in tandem. After all, it was all math. Automizing simple repetitive tasks with BASH scripts, statistical analysis of data, to numerical analysis in python helped climb the mountain of self-doubt. I was getting there.
But not quite!
There was always something new that would crop up, either on Stack Overflow or Reddit. I was developing a sense of contentment with what I had learned in the past few years, but this last decade has been the decade of software. It is all about services and soft tech. Where did my knowledge of alloys and thermodynamics stand in the market? Personally, I believe nothing is greater than the knowledge of thermodynamics, God of all sciences, if I may. But the market! Money. Food. Clothes. Hand-sanitizers. All essentials in life.
I could pride in my knowledge of hard sciences, but it felt one-sided. The world seemed to need me lesser and lesser, thanks to ML-powered materials databases. The machine could replace me, and would perhaps do so soon.
Nov 2020. I was trying to fit non-linear equations to a misbehaving dataset:
For hours I struggled with floating-point errors, division by zero errors, this value error and that. It was frustrating. I was about to pull out the hair from my head. It physically hurt to think I couldn't fit a curve. Finally, I started to doubt the theory itself. I checked and rechecked and hypothesised. Nothing worked. Not once did I doubt scipy.optimize.curve_fit. A half a century old thermodynamic theory I doubted, but not python. No.
The solution was an eye-opener. The theory was correct. The equation I typed was correct. scipy, too, was correct. It was the initial guess values of the fitting parameters that were the devil in the dark. I hadn’t provided any initial guesses since I didn’t know what to expect. scipy tried ‘1s’, which led to most of those errors. Simply providing a better initial guess value solved the problem. Instinct won. Not the algorithm.
I didn’t need to spend hours trying to doubt age-old theories. I didn’t need to be a prodigy in programming. Programming can get all fancy, but nothing is ever going to replace classic engineering and science. Intelligent databases, machines, dystopian future scenarios, come what may, but I will always stand tall for who I am. I am a materials engineer.
