Sign In
Get Clay Free →

Suggestions

    Joydeep Sen Sarma

    Co-Founder/CEO Clearfeed.ai

    Joydeep Sen Sarma is a well-known figure in the technology industry, currently focused on building ClearFeedAI while also serving as an advisor, mentor, and angel investor.

    With a strong background in computer science from prestigious institutions like the University of Pittsburgh and the Indian Institute of Technology, Delhi, Joydeep has been a part of various esteemed organizations over the years.

    Joydeep's journey includes roles such as Co-Founder and CEO at ClearFeed, Advisor at NextBillion.ai, Investor/Advisor in his self-employed capacity, Technical Advisor at Bidgely, and Co-Founder and CTO at Qubole.

    His professional experience also spans significant positions at tech giants like Facebook, Yahoo Inc., NetApp, and Oracle Corp, where he showcased his expertise in data infrastructure, architecture, and software engineering.

    Joydeep Sen Sarma's notable achievement includes founding Apache Hive, a widely-used data warehouse infrastructure project, showcasing his innovation and leadership in the tech community.

    Highlights

    Jan 18 · twitter

    Comrades, we must support E30. There is too much supply of Ethanol that must go somewhere. https://t.co/NbY90VFf1k

    Comrades, we must support E30. There is too much supply of Ethanol that must go somewhere. https://t
    Sep 17 · twitter

    How a Software Builder looks at LCA Tejas

    It turns out building and upgrading complex software - seems to have something in common with building and upgrading complex aircraft. A short story of things I learnt building cloud software for a long time.

    Story is based on Company "Q". Company "Q" had a very important cloud orchestration software. It was named after a porn magazine "H" - because - well why not. It was the Hadoop era - and "H" was a good omen of a starting letter. I coded up first version of "H" in a dinky store-room-office and had no idea it would become so important. Forked an OSS, hacked it, turned off all tests (obviously they wouldn't pass) and pushed to prod. It only worked in AWS.

    "H" became the heart of Q's software and the source of much differentiation and the crux of why people paid for our service. It routed an existing OSS in its category - partly because it rocked on AWS. The OSS competitor built for "multi-cloud" - but there was no multi-cloud other than Rackspace whose APIs rarely worked. "H" was serverless before that term was invented. It took AWS about 6 years to emulate the basic functionalities of "H" (that I had built in a few weeks).

    Fast forward a few years though - "H" had become an ugly hot mess - as any software without type checking and unit tests could be expected to when being hacked on by a now dedicated team of 3-4 engineers. What to do now?

    Luckily, "Q" now had to port it's service to new clouds - Google & Azure. An obvious path was chosen - let's rewrite the software "H" for new clouds, and at some point we would junk "H" altogether. So initiative "CM" - a complete rewrite of "H" - was born.

    "CM" started off well. We were able to get it to run on Google/Azure - but business there never took off much. All the $$$ came from AWS. And along with $$$ came REQUIREMENTS. ie. "H" kept growing and growing and growing. It was unkillable.

    One chap working on "CM" kept trying to make it catchup to "H" - but he was outgunned. And "Q"s executive team had no appetite to take on additional risk for a business that was already struggling. Everytime hard calls came to replace "H" with "CM" - it neither seemed feasible nor justifiable from a risk pov.

    Cut to the end - "Q" ended as many ventures do - as an equation on some PE guys spreadsheet. "CM" never replaced "H". "H" still lives, somewhere, and continues to get $$$ for whoever now owns the software originally-named-after-porn. (Even the porn mag went out of fashion - the internet killed it).

    So what does this have to do with Tejas?

    • "H" is "Imports" in IAF's fighter inventory. First Mig-21, then some M2K, now Rafale.

    • "CM" is the LCA. The prophesized solution to "H" and "Import Dependence". But always 3 steps behind.

    • "Q"'s Engineering team was like the IAF - we could never freeze the requirements for "H" (and hence for poor "CM") - because customer requirements and enemie weaponry don't freeze.

    • Exactly like the LCA program - "CM" took way way longer than we thought it would, the "imports"/"H" proved impossible to kill and "CM"s requirements kept changing as "H" moved faster than "CM" could catchup.

    The analogy - of course - is not going to be perfect - but an indulgent and wise reader might get the point. The lessons, in retrospect, were very clear:

    • If you want to replace something complex that also needs to evolve rapidly, put double the required energy on it. A Manhattan project of sorts.

      ie. - in IAF context - the IAF ain't gonna reduce import dependence by playing slow catchup. The world is moving too fast. Need Top $$$, Need top people from across the world. Sorry this DPSU shit with dinky dollars and quota culture ain't gonna work.

    • Try not to replace something very complex that fetches your daily bread. Its too hard.

    Now "Q" didn't have much of a choice here - but the IAF does. Mainline fighters, backbone of the force - are a very hard thing to not be competitive on. But how about:

    • UCAVs
    • Drones
    • and everything in the unmanned space.

    this is stuff YOU NEVER HAD. Something for which "imports" are not even a competition. Pilots won't die if these crash (ie. you can iterate much faster).

    If you can't solve the "import dependence" problem here, you certainly won't solve it where the operational bar and urgency is much much higher.

    • Its always the top leadership fault, not the individual contributors.

      The IAF is just a player in this game, not the designer. Like "Q"s Engineering team that had to respond to customer asks by evolving "H" continuously - the IAF has to just keep asking more features, because, well J20/J35/PL-XX/...

      And like the lone outmanned chap that was trying to get "CM" to catchup to "H" - the HAL chief is a also a bit player. Neither controls the $$$, nor has the ability to even fire or discipline his unionized workforce, nor the ability hire the best and brightest in the world for million dollar compensation. (Same for ADA etc)

      This negative spiral of delay, more features, leading to even more delay - can only be solved by top leadership putting a Manhattan Project together. (in the context of "Q" - i was certainly responsible for not ancipating this negative spiral - although tbf - it didn't really matter)

    And that, dear reader, is an over-the-hill software guy's take on the Tejas saga. While "CM" will never displace "H" - I hope I live to see our Govt. put a Manhattan project together - and stream out something even half as good as those dreamy UCAVs we saw in China's defence parade.

    Joydeep Sen Sarma
    Add to my network

    Location

    United States