Remember how I said I love my job, when I'm able to thwart the bureaucracy enough to actually do it? That's pretty much what I'm talking about; though because I'm at a higher level, I've actually got more layers, more forms, and more questions to deal with.
Here's a brief overview of what Bruce had to go through to get four (one each for development, testing, staging, and production) Windows-based Web servers:
Week 1 - At the same time Bruce's group started the project he went to procure the servers. He was told that all he needed to do was put in a Service Request with the Windows Server Team and they would get what we needed. However, that request is cancelled because when the Windows Server Team saw that the servers were for a new application they said, "Whoa, you have violated rule #38,991 of the Mega Bureaucracy! New applications must go through the Process for Application Implementation and Navigation."
Bruce starts into the fill the first two PAIN forms (one being 20 pages long with 150 questions), sends them off to the server team, and immediately receives a response that, no, do not directly send PAIN forms to the group they go to. Instead, open a project with the Mega Bureaucracy's project tracking system, attach the forms and THEN assign the project to the group.
A few days later, he receives word that the project has been accepted, slotted, and a project manager assigned. Bruce figures, "Cool, now we are moving! I'll have my servers in no time!" He and his boss have a conference call with the PM and express to him the time critical nature of these servers. The PM agrees to push them forward saying that the request isn't complex and shouldn't take much effort.
...Snip...
Week 18 - The head of IT for our division finds out that we are still waiting. Heads start rolling...even poor Bruce's. "WHY DIDN'T YOU CALL ME SIX %($$!*& WEEKS AGO???" the IT head blasts.
Week 19 - The servers are built (it only took 2 days to build them!) and are signed off for production support.
Week 20 - Bruce distributes the application URL pointing to the brand new servers.
Through all of this Bruce learned a couple things. First, don't even think of going around the Mega Bureaucracy, even if somebody says you can. The Mega Bureaucracy remembers and brands you a heretic. Second, if you think you will need help from the Mega Bureaucracy, start early, fill out all of the forms, stand in the right lines, sacrifice to the appropriate gods, and don't even hint that you would think of going around them. Finally, he who yells loudest gets move the front of the queue soonest - as holy and almighty as The Mega Bureaucracy is, they're happiest to get rid of their crabbiest customers first.
The silver lining in all of this? Apparently, the Guardians of the Mega Bureaucracy seem to now be willing to consider that there is a different tier of requests that don't require so many stopping points, designed to make sure that users really, REALLY know what they want to request. Bruce remains positive saying that, maybe in a few years, after meetings to plan meetings, forms to request forms, they will have a process that only has an initial questionnaire of 10 pages and 75 questions.
Oh and 20 weeks start to finish for four servers? That's fast. I've had similar requests take 18 months before.
We do have an "accelerated" process that's supposed to take 10 days... and it does... 10 days from the forecast install date until the server it turned over to testing. Then two weeks in testing before it's handed over to you.
Of course you have to be on the forecast 30 days in advance of your planned install date. And to get on that forecast you have to project the funding and capacity request six weeks to three months in advance of the forecasting date; and get enterprise planning approval from your organizations enterprise planner.
It's all very complicated.
Say you have a need for computing resources to run your application. What you do is you go to your enterprise planner and let them know you need computing resources (if you have a very good idea of what you need already, including approximate costs); or before you go to your planner, you come to me and say "I think I might need some servers".
Either way, the next step is to consult with me and my team. We work with you to figure out your business and technical requirements. Then we architect a high level solution to fit within enterprise standards, practices, and direction; or if necessary get exceptions to those standards, practices, and direction. We also decide if your solution can be met from our standard catalog of preconfigured (not really) offerings, or if it has to be a custom configuration.
If you can use a standard offering, then you MAY be eligible to go through the "10 day" process. Otherwise, it's the "custom" non-expedited process for you.
At the end of that initial consultation we give you a high level architecture, and a guidance number, that will be your budget request to within +/- 20%, and the core of your business case.
Then you go back and write up a business case; and take it to your planner, and your management, with that guidance number; and ask for the budget allocation and to be put in the project pipeline.
If your planner and management give you approval, then your project is opened by your planner and the project manager for your organization, and the timeline officially begins.
This could be any time from the day after you ask your planner, to 12 months later by the way.
Your first step in the project pipeline is back to my team; where we re-evaluate the project and account for any changes that may have occurred since we first consulted with you. Then we take the high level architecture we originally created, and refine it into a detailed architecture and design. We also get guidance quotes from any outside vendors, and internal quotes from all the supporting groups; and produce a costing document accounting for all hardware, software, support, infrastructure, facilities, and labor costs, extended out for four years.
This will be your final budget number to within 10% unless there are changes to the project made during the implementation process.
Finally, we all meet together to ensure that the proposed solution meets the business needs of your group, that it is within budget, that it fits on the schedule, and that it meets enterprise standards, practices, and direction; or that exceptions for such have been approved.
You sign off on it as the project requester, the planner signs off on it, and as chief architect, I sign off on it; certifying it as ready to go.
But wait, there's more.
Up 'til now, the clock isn't running. Once we sign off on it and get ready to kick it into implementation, this is where things get REALLY complicated.
The first step is funding approval (finance departments for the enterprise, and for your organization; as well as your management and your enterprise planner), hardware approval (hardware finance), software licensing approval (software finance).
We can't move to implementation phase without those; and they aren't covered by all the previous approvals.
For this process the clock IS running; and it's a 7 day clock, that can be extended out to 30 days. If that clock runs out the project is canceled and you have to start over.
Once you get the funding approvals, THEN the actual implementation phase begins, and we hand the project over to the Systems Integration department, who manage the implementation and integration process.
So then you get that 10 day server build thing right?
Well, no.
First, the project has to go through hardware engineering (the guys who write the build sheets and parts lists), and the output of the HE team goes back to ME for my approval (to ensure it matches what we specified in the design and architecture phase). From there, it goes back to the vendor for a final quote using the parts specified in the HE build sheet. That quote is sent to contracts to for contract approval, and then on to purchasing.
Then there's the network infrastructure approval (network engineering), security approval (security team), firewall approval (firewall team), hostname and IP address allocation and approval (DNS management), datacenter approval (datacenter engineering),power approval (power committee); before systems integration hands it off to the build team for your 10 day build.
And remember, that's the expedited "10 day" process. The groups involved in all the above have agreed that once the clock starts running at funding approval, that their inputs and approvals for projects that have been approved by my group to go through the expedited process, will be completed within the 30-45 day forecasting window (depdning on what exact date you submitted the request).
So when it comes down to it, the "10 day" process, is actually 6-13 weeks plus 10 days to actually build the box (it usually only takes a few hours actually, but they build it within a 10 day window), plus 14 days for testing.
The non-expedited process can take any amount of time at all; in my experience up to two years; so long as it doesn't take more than 90 days in any one stage (including each individual approval), which will trigger automatic cancellation and you have to start all over again.
Of course, it is entirely possible if someone with enough clout, shouts loud enough, or bad enough things are happening; to have 24 new $300,000 boxes and 20 terabytes of enterprise SAN storage, on the floor, built, tested, and ready to go in 7 days.
It's all about who's yelling, how loud, at whom.
Oh and all that stuff? My team does about five hundred of them a year; and that's actually only about 1/4 of my job responsibilities. The other stuff is even more convoluted and takes even more time.
Aint big business just grand?
