Wednesday, November 17, 2021

The trouble with Citrix

It  looks for all the world like Citrix... One of the more important technology and services companies of the last 25 years...  is in the middle of imploding...

... As many have expected would happen for quite a long time now...

...Not with a bang, but with several extended half hearted whimpers; a lot of corporate doublespeak, and a lot of very angry and disappointed people.

I  still have a fair few friends working at Citrix... good people all of them, who for some reason had faith that the companies management couldn't possibly stay that bad and that dumb for that long, and somehow they'd reverse course. 

They've been trying for years now, to get management to put REAL effort and support and investment and energy (For more than a couple quarters without reversing course or redirecting or "refocusing" or otherwise making it impossible not to fail) into the remaining useful and long term viable core technologies and solution sets; and into the interesting side developments with potential to become viable core, or substantial secondary or peripheral technologies, products, and solution sets...

...Critical things, that so many good folks have tried to make happen, and tried to make work,, and done their best to patch up around and make things work anyway when they didn't happen... for the last 10-15 years....

And of course, by far most importantly: they have tried their mightiest and best, to get management to invest in, develop, and fully support; all the PEOPLE absolutely necessary to make those technologies, and products, and solution sets; viable, and then successful,, in the marketplace. 

The people to support enterprise sales, and to develop, implement, and support enterprise solutions; to be useful, successful, and EASY to integrate into the operations of every organization; across the breadth of the multi plarform, multi-environment, mixed local and remote, mixed physical.and virtual; mixed on prem, and hybrid cloud, and public cloud, and managed service, and infrastructure and software as a service environments... And every other possible environment and technology, and sector, and solution set... 

Which is the only way that those technologies and products and solution sets... and Citrix as a technology and services company...  can compete and be viable in that marketplace

The people, and the services they provide, that should be Citrixes REAL core business, and biggest source of revenue, and profit.

...But thats not happened so far... and Probably won't in the near future...

In fact... rather the opposite Is happening right now... and has been happening for... way too damn long now. 

Ok...this is going to be a long one, because it needs to be, to illustrate the totality of the... it's not a failure as such, as it is just a total abandonment by management of an entire business, because they didn't know when they needed to change, and how they needed to change when they did... And just couldn't be bothered...

... The problem with Citrix...

A lot of folks have made sage pronouncements about Citrix like "Oh they stopped innovating, and once tech companies stop innovating they're done for"... which...yeah it's generally true, but it doesn't really capture the scope, breath, and depth of the issue...

Citrix didn't just stop innovating... Their business model stopped being viable; because the core technical and business operational functions their products and solution sets  performed for their customers; either became no longer necessary or relevant; or they were eventually wrapped into the base functionality of the platforms and systems they used to provide those functions for. 

Virtualization, remote access, published virtual desktops, and published virtualize applications; are now all included for free or very low cost, in every commercial server operating system or hyperion platform. 

Virtualization, remote access, published virtual desktops, and published virtualized applications; are now all included for free or very low cost, in every commercial and most free  server operating systems, and in every hypervisor platform (again, whether commercial or free); with enterprise class support commonly available at relatively reasonable premiums. 

...Sure, Citrix generally provides a lot of additional functionality... usually in the area of enterprise management and support features, and actual enterprise support... that the stuff the other companies are giving away for free does not (or only does so in a more limited way)... But citrix ain't cheap, and it's awfully hard to compete as "expensive, but somewhat better" against "it's free and it works"... Especially when the product has become less and less "better" over time...

...And frankly... Citrix as an entity, and their core management, have really never dealt with this fact, never mind managing to find or invent a new business model for themselves... They're STILL trying to sell at a hefty premium, what everyone else is giving away for free... 

...and in fact which they ALSO give away for free, in a somewhat decontented, and of course entirely unsupported form... With the message that "you already have the skills and knowledge and configurations ND tools to manage our stuff in your enterprise... why would you risk changing?"... A message that... Let's just say hasn't notably worked out for them so far.

Then, instead of figuring out a new way for a new business model to work, with the technology and people and intelligence to back it; Citrix management just tried to kinda tweak the old model, and announce BIG BOLD NEW things, that really weren't... They were just the old things repackaged and relabled and rejiggeed to sort of do something different from how they did it before, or to sort of replicate some other companies or some other platforms functionality... "Now in that good old citrix color and flavor you love" Never mind trying to actually do something new, or better, or otherwise provide some kind of actual competitive advantage and reason to chose their solution over others..

Then, instead of figuring out a new way for a new business model to work, with the technology and people and intelligence to back it; Citrix management  just tried to kinda tweak the old model, and announce BIG BOLD NEW things, that really weren't... They were just the old things repackaged and relabled and rejiggeed to sort of do something different from how they did it before, or to sort of replicate some other companies or some other platforms functionality... "Now in that good old citrix color and flavor you love" Never mind trying to actually do something new, or better, or otherwise provide some kind of actual competitive advantage and reason to chose their solution over others.. 

And just like they sorta kinda copied other vendors products and solutions... because "hey, the other guy is selling them pretty well,, we should be able to too right?"... they sorta kinda tried to  copy the business models of SEVERAL other vendors...including multiple times trying to execute on multiple contradictory and mutually exclusive models all at once...

...Really...At least from the outside anyway... It seems like management just sort of closed their eyes,, crossed their fingers and pointed their heads down; and hoped that their (once huge but now rapidly and continually shrinking) legacy installed base and relationships... and basically, inertia...would carry them through...

Then, even once it was completely impossible to not know that idea... which could never have worked to begin with...  was inevitably and unalterably failing; they continied to ignore reality, and pretend otherwise. Because exec bonuses maybe? Can't think of any other possible reason, unless those supposedly smart and successful people really CAN be that dumb, and that blind, for that long...

How bad is it, and how do I know it's that bad?.

I have spent almost the entirety of the last 25ish years, working variously as a Citrix customer, partner, or otherwise a contractor or consultant...

...Specifically in roles as presales, post sales, internal, and independent outside:

-- Solutions architect, solutions delivery team leader, and customer success team technical manager.

-- Infrastructure architect and operations manager (traditional on prem, mixed on prem and third party co-location; utility compute and high reliability exascale HPC and OLTP, on prem, distributed, and hybrid; hyperconverged, on prem, distributed, and hybrid; hybrid cloud, full private cloud, and full public cloud... In traditional, DevOps, MSP, and every other kind of operational concept).

-- Service delivery architect and operations manager (including traditional, devops, MSP, and every other kind of perational concept)

-- Information security architect and operations manager (traditional, devsecops, and otherwise); assessor, and auditor.

-- Policy Process and Audit, and Governance and Regulatory Compliance; architect, manager, assessor, and auditor.

-- Disaster prearedness and recovery, business continuity, emergency response, and incident response; architect, operations manager, tactical/emergency/response team, leader/coordinator; investigator and forensic analyst; and mitigation and return to operations coordinator/team leader. 

... And I'm going into way too much detail and way too deep inside baseball, for a specific reason that I will get to in a minute...

...Almost all of these in high risk, highly regulated environemnts, including government, defense, medical and pharmaceutical, and financial; sectors, and legal and regulatory regimes and frameworks

I said all of that, to make it absolutely clear just how badly, and at what scale Citrix have failed in... basically everything they should have been doing for the last decade or more really.

Because, since 2009, in all of the roles I have mentioned above... 

...All areas where prior to about 2005-2009ish Citrix had dominant market positions.

And they had those dominant market positions, in large part, because although they may not have had the best or the ideal technology or implementation out of the box; they had in house (and local regional in every region, including specialists for different markets and industries or sectors, and specialists for unique technical, operational, or legal/regulatory requirements) solutions and subject matter experts, and pre and post sales team support, including implementation and customer validation/UAT support before handing over to a sustaining engineering and support function; and real enterprise customer support from the help desk on up to real support engineers and sustaining engineers; who understood all of the above, and could be counted on to make things work, and solve problems with the customer, or reseller, or end user etc...

You had to pay fairly dearly for it.. But it when you did, at that time, it was worth it. They gave good value, and they made it work, often when others could not. 

Hell.. prior to 2001, all of those markets I mentioned above... all very particular and challenging spaces to compete in... were just THEIRS period. Or at least they were if you had to deal at all with Windows or cross platform, cross environment, remote or published resources, in interactive graphical sessions... which almost every big company and a hell of a lot of small and medium sized companies did... and still do... to some extent or another. 

... But since between 2005 and 2009... almost 17 years at the outside, and almost 13 years at the least... 

...and across all those roles and environments and segments etc... All of which are core mission spaces for Citrix, and critical to the future of any company wanting to operate therein...

I have only seen TWO actual  completely new, de novo blue sky, major implementations of core Citrix solution sets, in critical and core operational functions for the organizations in question.

Two... Yes, really... TWO...

In EVERY OTHER CASE... and we're talking hundreds of clients and customers and partners and employers; across thousands of sites... any growth in Citrix installed base in those organizations was essentially by default, or forced, or something they settled for because of mass and inertia and long term contracts; or was otherwise just something they had to accept, because for whatever reason, they had no other viable option, or any other option was just too difficult or not worth the effort etc...

In almost all cases they were either already in the process of getting rid of Citrix as much as they could (and often already working AROUND their existing Citrix solutions footprint);  investigating the idea and what it would take to move to an alternative; or doing competitive evoluations of alternatives; or at the very least they WANTED to stop depending on Citrix solutions; just that for whatever reason they were stuck for now, and once they could, they'd be replacing or just removing Citrix as soon as it was practical to do so.

And dozens of them did... They either completely ripped Citrix out of their operations entirely, or they reduced them to minimal footprints, of things they absolutely couldn't get rid of or get a viable alternative for... and only until that limitation changed. 

A lot of them TRIED to expand their Citrix solutions footprint...  lot of them actually DID... AND AS I said there were exactly TWO large enterprise customers, with actual brand new, didn't have at least a moderately significant relationship with Citrix before,  major core functionality solutions sets; purchased, architected, implemented, and put into operation...

...And both of them had projects to minimize dependency on Citrix, or rip and replace them completely, within two years of handover on these multi year, mulmillion dollar evaluation, architecture, implementation, and testing projects....

I saw seat counts for existing or replacement products/solutions grow, and expansions to existing products and side products piggyback off them... Almost always in organizations with major enterprise licensing and discount agreements and large longstanding preexisting Citrix footprints... 

And occasionally, though rarely, I'd  see new implementations of small solutions and systems... a few systems, with a few seats, for one or two products... Not major critical or core operations, or just for one or a couple  critical but segmented off things, that were effectively one offs and exceptions, and Citrix was either the enterprise default solution, or for some reason it was either the cheapest solution (again usually an enterprise license agreement), or it was the only solution that supported a specific oddball edge case technology, product, configuration, process etc... 

But that was it... It was as if they were running in place.... Or worse, running backwards...
Suee, they still had a lot of legacy customers... Whether it was because the customers had a huge installed base already and couldn't  afford the license costs, training costs, support costs, down time etc...to move off of Citrix. Or they had specific contractual, legal, regulatory, supportability, or edge case technical  requirements that prevented them. Or for whatever reason there simply wasn't any viable option to not use Citrix...

...In every single case, none of them wanted to be dependent on Citrix anymore, and most didn't want to use any Citrix solutions period...

Why did they not want to use Citrix anymore, after years... sometimes decades... of relying on them?

Mostly, it was because for the last decade or more, the experience of having to rely on Citrix, became more and more unpleasant for them... and less and less reasonable, supportable, viable, or even tolerable. 

Sometimes, the product or solution was literally permanently and unfixably broken and they couldnt use it and had to buy or build and implement workarounds, or entire alternate systems.

 Sometimes it became a bodge job of just barely functional barely supportable chewing gum and baling wire, but they couldn't make anything else work. 

Sometimes  licensing had become so stupid and harmful that they couldn't or wouldn't pay for it anymore.

Sometimes It was because the previously excellent support for weird solutions, edge cases,, unique environmwnts and requirements etc... disappeared, and they were replaced with offshore script readers and box checkers... 

Whatever it was, over the past more than a few years,, there have been myriads of compelling reasons to NOT use Citrix, and not very many good reasons to do so. 

...And then, they just stopped even pretending to try to have a viable plan or technology moving forward, and for some indecipherable reason,, decided to lean into the fact that they were now IBM from the 70s, after cannibalizing their own channel and their own customers, they were just going to try to minimize costs and roll those existing customer seat counts and license revenues up year to year...

...And that's kinda the ballgame in this industry folks...

That doesn't mean there aren't still plenty of good people hanging on...  But that doesn't matter anymore ubfortunately... sadly... 

Because a company with no business model; few relevant products they can actually sell into relevant markets, and operationally support for relevant customers requirements;  and few, or zero, core technological, functional, operational, financial, or otherwise in any way compelling reasons, for any potential customer to select that companies products, solutions, and services, over those of any other vendor... 

...Except perhaps by inertia, edge exception, or ignorance of other options... 

...They aren't really a technology company anymore... They're just a contract servicer, cutting whatever costs they conceivably can, without impeding what is really the only meaningfully functioning part of the business: collecting whatever "dead money" revenues they can from their existing contracts and customer base, for as long as they can; making their quarterly earnings per share appear to be as high as they can plausibly make them appear... 

Thus, "executive management" can squeeze the most possible cash compensation for themselves out of this one time great company; before finally breaking up the rusting hulk, to extract any last pennies of value, from whatever bits and pieces anyone might offer any money for; of the remains of what used to be Citrix.