Improv Service Provider

What internal services within an organization can learn from improv comedy.

From experiencing organizations ranging from 10-person start-ups to dozens-of-thousands-of-people organizations, I have found a consistent experience among shared service providers. They say “no.” A lot.

They do this for a lot of reasons, the main one is protecting their job satisfaction and quality of life. The organization exists to pursue some goals such as improving developer’s lives, providing a fundamental platform for businesses to operate, installing infrastructure to provide connectivity to millions, fixing printers at non-profits around the capital region, or defend the country’s physical and technological borders. These goals are something that a small population of people actually do and a majority of the workforce does things that enable them in various ways.

This analogy is likely to fall apart for certain supporting roles, such as accounting. I am not deep in the field of accounting but GAAP and compliance rules are tougher to work around.

For everything else, let’s look at how an organization would typically respond and then compare that to an improv approach. We will look at Strawman Inc. A company that is completely fake and simply reflects intra-organizational patterns that consistently pop up across my career as an employee, consultant, contractor, and spectator.

Strawman’s IT Department

A new employee joins the company and requests a Macbook.

What Strawman IT knows:

  • We are a windows shop
  • Our GPOs require windows
  • Our file shares are windows
  • Role is a middle manager
  • Policy includes 3 laptop configurations, none are this

What Strawman IT doesn’t know:

  • The new person is part of a strategic initiative to pivot a languishing market segment
  • Mac hardware is required to make an iOS app which is what they are hired to do
  • There isn’t a high-enough-paid-role to bring in the talent as an individual contributor, hence the weird title

Obviously, the solution here is to learn these things and figure out an SLA that the IT team can support. Knowing this is a pivot and the point is a new organizational capability makes it clear, but what if it’s just a new person who doesn’t like windows? What if it’s someone without technical experience who would just be doing spreadsheets and MS project all day and complaining that the mac office suite isn’t keeping up?

DENIED

So at any large organization I’ve worked at, this request would be outright rejected. The escalation process in some cases would catch and allow the discussion. In other cases, the escalation process would create a terrible relationship with the new person.

With the request denied and the windows laptop provisioned, the new employee has to outsource development or create their own shadow IT macbook. Outsourcing a pivot rarely works and shadow IT is a huge risk for the org where the employee will be exfiltrating data, code, business system content, and other things. In either case, the organization is setting the initiative up for failure.

Notice that it’s the organization failing and not the IT department. The support personnel would be taking a huge risk if they empower the user to succeed without buy-in from the organization. This is a system failure and requires following a systems-thinking approach to solve.

YES AND

Rather than denying the request outright, the optimal solution is to feel out whether the person actually has the support of the organizational leadership to make changes that impact the system. The system will fight these, every individual will dislike changing the scope of work they do. What does yes and get you here?

  • A process that productively gathers exceptions
  • Feedback on the existing policies to refine and optimize
  • The organization can pivot or deliberately decide this isn’t the right pivot

How does it play out?

Assuming the same request, the support personnel who gets the provision a laptop ticket sees that it’s outside of the approved spec and responds internally within the team by filling out a yes and ticket raising their concerns.

We can provision you a macbook and it will be a second-class device. This means that business system connectivity will be restricted. To work around that, here’s VDI or a desktop PC that you can remote into. For endpoint management, the organization will have to buy a tool like JAMF and set some restrictions. The support team will need another support person to be onboarded to cover ticket load while the team rotates through Mac and VDI and JAMF support trainings.

The feedback here has some pretty significant costs and risks associated with it. The pivot initiative likely has a budget that would be dedicated to changes like this. If it doesn’t, then the discussion can happen now with some steps. You’ve probably caught the cascade here.

The yes and request is going to hit procurement, recruiting, training, and those requests may be answered with no which will sabotage the outcome the same way as the initial IT response would. Each of those teams has their own set of risk aversions and learned helplessness issues to work through to be able to support the request. The Toyota Kata solution here would be to swarm the request with business analysts and trace down the impacts throughout the organization quickly, decide how to proceed, and then monitor the situation for a bit. The GitLab way to do this would be establish a DRI, collect feedback in a Merge Request that changes the handbook, and then the teams impacted would agree or dissent constructively. Many other orgs would just hit an intransigent silo and the initiative is hobbled.

Also, for the record, GitLab experienced the opposite request for windows laptops from employees and rejects them deliberately for reasons while still having an exception process.

Strawman’s Application Security

We can reuse the same example since there are 2 components, the endpoint and the purpose. This request is after the new employee has created a prototype and needs to deploy it.

What Strawman’s App Sec team knows:

  • We are a windows shop
  • We deploy 96-core xeons in our datacenter
  • They’re all used so buy some new hardware
  • We require 5 environments in separate networks, so 5x the order
  • The software needs to comply with .NET security scans
  • The software is Swift and NodeJS so it can’t be deployed
  • All databases need to be on “the database cluster” for availability and security
  • It’s an oracle DB so license costs
  • Authentication needs to go through an old provider that doesn’t support SAML or OIDC

Obviously…

DENIED

The security team exists to limit risk. They can’t make software so they just provide advice and when software violates the policies, they put up a road block. Much of the software that is deployed was built before the App Sec team was formed and is rife with vulnerabilities but since there’s no visibility it’s largely excused. And it’s mostly going to be decommissioned as soon as the new software is running which is all secure. So this request is too far afield and the service component never be deployed until it’s rebuilt in .NET Framework 3.5.

Is this strawman too insane for you? It’s literally happening every day. It happens after millions of dollars are invested in these initiatives. The road blocks are eventually cleared but in a way that requires a business champion to force the security team to back down. At this point the security architects and penetration testers give it a not allowed but okay and tries to wash their hands of the security consequences. In reality, they don’t have that luxury because if there’s a huge breach, the company suffers and the security team with it.

YES AND

Rather than denying the request outright, the optimal solution is to have already been involved when the planning happened. When the developer runs yarn dev for the first time or builds the first iOS package, the security team should be right there, learning about the requirements and capabilities. If they have a good sense for the frameworks and where security-by-default is already baked in, they’d be much more comfortable with the newer tooling. Having a parallel process in the business space working out what the risk appetite is, what systems will be accessed and how, where other infrastructure and technical debt will impact the plans, and anticipate any other impediments that are looming.

We can provide a security architecture analysis of this project and it will lead to actually decommissioning the legacy services but the level of effort will be much higher than usual. The level of effort is unknowable until the team is engaged so we need to put a hold on another type of change that is always problematic. The standard hardware doesn’t make sense for this initiative so evaluating cloud offerings, serverless functions, nosql databases, and message queues will enable the new software to scale effectively.

Starting with the notion that this step is pivotal in being able to actually follow-through on the decommissioning plan is really the core to this. Once a gigantic cost and drag item is tied to an initiative, the budget can be set as an ROI against specific pain the organization feels. This would likely create two factions, those who hate interacting with the legacy systems and those who support the legacy systems. To win over the support personnel (who want to be won-over, simply rotate them into the new system deployment efforts. NodeJS servers still need support, or better yet, cloud stuff!

Next steps

There are a few deliberate steps you can take, regardless of whether you’re in a position to enforce these rules or are subjected to them.

The short answer is rather than saying “no”, answer any even moderately reasonable request with yes and while including a way to achieve it, even if outlandish, that they’d have to agree to.

Obviously without control of everything in an organization, this strategy will play out as yes and followed by “sorry, my manager says I can’t.”

For the individual contributor

If you’re part of a support organization (meaning that your users, clients, customers, sources of work are internal to the organization), determine if you are participating in these system defects. You can tell if one of the following is true:

  • You have no customer satisfaction type of feedback
  • People try to do things without reaching out to your organization
  • The outcome is often denial “due to policy”
  • Your team is made to do weird things becomes some executive makes it happen

The same list of items that lets you know something is wrong can be used to fix the problem. Solicit feedback from people who interact with your organization unofficially. Hold a weekly or monthly retrospective on the team’s activities. If you can’t get your manager to do it, do it informally. The records speak for themselves so you don’t need to criticize any individual’s work. Remember these are system failures and need to be framed as such. If your retrospective results in “Frank is ruining the organization” it probably wasn’t effective.

Retrospective outputs should be a count of exceptions that were handled well, a list of those that were handled poorly and an attempt at identifying the cause, and then a review of customer satisfaction feedback. The last thing is a brainstorm on some things that can be tested in the coming weeks to try to improve any aspects. Keep the experiments small and focused don’t do too many things at once so you can sense which things help and which don’t. Keep the things to try within your team until you can get management and executive buy-in.

If you’re part of the mission team that doesn’t support internal customers, raise concerns when you see them. Being courageous is required in many organizations to voice frustration since the default reactions are defensive. Criticizing someone else is a great way to lose them as an ally in the coming struggles. To be able to provide effective feedback, focus on the system failures and point out how the people who couldn’t help you were constrained by things beyond their control. If you can identify those things, do so. If you can’t, ask. People typically like talking about their work if you’re just asking questions.

Using the system traps may help people who are interested in improvement see what your approach is and why you’re taking it. Don’t take it personally when met with resistance. You can be sure that everyone who is creating and enforcing these blocking behaviors is doing so because of painful lessons they’ve learned over their careers. Some at this organization, some they bring with them. At some point, escalating to a manager may be necessary but avoid bringing the escalation as a personnel complaint and instead “support the employee” by trying to unblock a better workflow.

For the managers and executives

Be aware that raising these concerns is an act of courage and if you’re hearing one issue, it’s likely impacting many more silently-suffering types who just want a paycheck and don’t find the efficiency gains worth the trouble. The majority of employees are just that so if you give them their salary to work at 30% efficiency, they’ll do that since the organization controls both the salary and the pain. In a start-up environment, where people have a stake in the success of the team, there is a lot less courage required, a lot more autonomy and sense of ownership. Trying to inspire that in a bureaucracy isn’t within the scope of this article, but it’s a good idea.

The problems fall into a few categories:

  1. Identifying the problems
  2. Get free from learned helplessness
  3. Implementing fixes at the correct leverage points

If you haven’t noticed, all of these things are activities that the people at the bottom are actually best suited to do.

For 1, be involved in the exception handling process and if there isn’t a retrospective that tackles the exceptions, start running one. It can be asynchronous if your team is small and busy, but be sure that all denied requests have a way to be accomplished and what problems really stop it from moving forward. Bring these to cross-team discussions and talk through them in a wargames format to see what it would really take to get them done. Try to get other managers to pretend like the request is an absolute must due to a legal requirement or else the organization dies. How would they do it? What would it cost? Highlight the places where unknowns that may not actually be severely costly or risky are sufficient to kill an idea.

For 2, empower your team and other’s teams to speak up about things they’re seeing. Create an environment where things that are called out are appreciated and rewarded. There are different approaches to this, but any psychological safety papers or books are full of great ideas. The conversations that come out of it are incredibly uncomfortable so take steps to overcome that with very positive sentiments at the end.

For 3, the cross-team collaboration component really lets the fixes go the right way. All system failures are that way because of multiple feedback loops which are serving their own incentives to degrade the overall organization in order to achieve some local optimum. Working within your team creates more local optimums but the system is everything else too. Your vantage point can be used to empower the individual contributors on your team to identify leverage points elsewhere in the organization and collaborate with the other teams. Your role would be lightning rod for complaints and escalations where other teams feel insulted or challenged.

But why?

I’m going to finish this lengthy post with the ultimate question.

As an [insert role here], I am doing what I can and getting paid so I don’t care about this.

This is absolutely true and a perfectly reasonable way to operate considering how employees are typically rewarded and punished. My audience for this blog are people who are upset by the “brownfield” experiences and want better. A lot of people who are playing a detracting role in organizations that are struggling, private and public sector, are doing so because they don’t particularly care. This seems to be a centralization-of-resources problem where start-ups and disruptors can thrive until they grow big enough to develop the same bureaucratic failings of the disrupted market organizations.

If you personally develop the mindset and skills outlined here, you can be a force for good. Any time an organization gets more efficient, the people doing the work can benefit from the efficiency by working on more interesting things or doing less work.

In the public sector space, it’s incumbent on everyone to treat the taxpayer’s money seriously and use the best available methods to get the outcomes the people need. System failures have created enough bloated and defective services that the citizens are losing faith in the institutions. If you are among these public servants who experience the frustrations on a daily basis, make noise. Reach out to other public servants. Every government employee and contractor I’ve met does want to do the right things and works for less income to serve a public good. They are also some of the most impeded because their mission-orientation lets the service organizations abuse them more.

Share stories and tweet

This is the last thing… We need to normalize calling out system deficiencies and avoid personal attacks. When you experience some terrible customer experience inside your organization, make noise in the company slack or email or break room. Hear the corroborating evidence and elevate the discussion to how expensive it is for so many people to be wasting so much time and energy. Focus on using that energy for a better purpose rather than “I don’t wanna.”

As always, feedback is welcome.