Product Managers: Data-driven experiments are an excuse not to be brave
Product Managers: Data-driven experiments are an excuse not to be brave
Being “data-driven” is “virtual signalling for Product Managers” and not the best method for some product decisions and roles.
This snippet from a longer conversation with Patrick Collins of Zip.co digs into the suprising insight of different traits in PMs.
Patrick quipped about his experience: “I’ve only been doing this product thing for 10 or 15 years”.Â
So it’s clear he has a wealth of experience and carries a lot of lessons and scars from both web and mobile products.
Patrick’s advice to aspiring and existing Product Managers is to know which type you are:
- Analytical
- Intuitive and/or creative.
- Technical
In recent years he’s seen migration from Project Management and Business Analyst roles into Product Management. Also there is often a progression from developers into the PM track. But perhaps these new PMs are more analytical and don’t carry a sense of what a great user experience is like.
The concern is that:
“experimentation and being data driven,
can be a cheap out for being brave”.Â
In other words a Product Manager who has the courage to make big bets based on solid user experience background is invaluable in creating a “generational leap” in product quality.
Patrick notes that this trait is not based on seniority, its really that many PMs are risk adverse. They get into a process of “polishing the turd”.
It’s an interesting conundrum, in other posts we’ve noted the time expense of A/B tests to get statistical significance, the challenge of PMs wearing many hats and personality traits.
“But the idea that we can just test our way out of every problem is dangerous because it can really hold a product back.”
Like most human skills, its part-art and part science. Patrick says:Â “I still haven’t quite cracked the formula for what kinds of PM is able to know when to stop polishing and know when to go for a generational leap.”
“some being more creative, some being more analytical and some being more technical and knowing who you are”
The standard 3-legged stool for PMs is to balance UX, Tech, Business – you may have seen the Venn Diagram in other posts on this blog.
But the image above illustrated the best fit for PM’s who are pre-disposed to traits of “Technologist”, “Generalist” and “Business-oriented” – the Focus field is a clue as to the persons strengths and weaknesses and ultimately what would be the right product-component for them to work on.
Digging deeper into the downside of being too analytical, the obvious A/B problem arises:
“The experimentation culture, I think is damaging in some ways, because it comes across as this kind of analytical thing that that you can test your way out of anything. But it’s like, well, what the hell are you going to test? And how are you coming up with that?”
If you want to hear more from Patrick on a broad range of topics and experience, join me for our WFH Fireside chat this coming thursday. If you can’t make it, sign up to the blog and it will be in the upcoming posts.
A note about the cover cartoon
I was wondering about what picture to put up the top of the post. Perhaps “fork-in-the-road” to reflect product management career choices. Perhaps an “all-in-bet” pic from a casino etc.
But I actually found a post on Basecamp’s site under their ShapeUp series that captures part of what Patrick was covering.Â
Basecamp are often controvesial, always counter-trend and very, very authentic. This series says:
Shapeup is Shape Up is for product development teams who struggle to ship. Full of eye-opening insights, Shape Up will help you break free of “best practices” that aren’t working, think deeper about the right problems, and start shipping meaningful projects.
and that sounds pretty bloody good to me – I’m adding it to my reading list.
The image came from:Â https://basecamp.com/shapeup/2.1-chapter-07
Transcript
Patrick Collins 0:03
Learning of the different kinds of product managers, some being more creative, some being more analytical and some being more technical and knowing who you are. And therefore what if you know i’d imagine the audience is probably going to be aspiring PMs, some of them anyway. knowing who you are as a PM really will help.
Patrick Collins 0:27
And you don’t want to be a creative PM, running the platform services team, or tech or a technical PM running the onboarding project for the app, right?
David Jones 0:40
Why was it specifically what did you learn that specifically at Moveweb?
Patrick Collins 0:44
I guess I’ve learned over both of these past roles, that those different PMs can actually fail in one role and succeed and I don’t mean outright fail, like crashing and burn. I mean succeed in other ones. And so when I look for PMS, I look carefully at their background. And I look at what skills they have. And that will lean me slightly towards where I think they’re probably going to succeed as PM. Creative requires you to take some bold steps. And I think the experimentation culture, I think is damaging in some ways, because it comes across as this kind of analytical thing that that you can test your way out of anything. But it’s like, What the hell are you going to test and how are you coming up with that? Which part and you know, I think I’m on consumer app in particular, it requires a lot of bravery and a lot of courage to translate what you’re hearing from customers into a point of view, and then go test that and having that having that courage to translate data analytics and and customer interview data into into a point of view: not many PMS have that.
Patrick Collins 2:03
And it’s actually not a seniority thing I’ve noticed it’s a, it’s a risk taking trait that some PMs just will never get.
David Jones 2:12
So just to, just to play that back to you, if I’m a really good data centric type product manager, there was a Stanford saw talk I saw which was called “gut, data, gut”, which was that you’ve got to start with an intuition first, then you can go and get the data to support it or actually run and experiment for that data, then you actually have a new iteration. Are you saying that an analytic type product manager really just misses that sort of “gut” type thing and testing many different sorts of things?
Patrick Collins 2:44
CAN. Yeah, I think the concept of experimentation and being data driven, can be a cheap out for being brave and not taking not making courageous leaps on the product and maybe polishing a turd, so to speak. And so sure, I think there’s a really difficult line to walk between knowing when to polish and knowing when to try to “go for it”, like a generational leap. And it’s still kind of I’ve only been doing this product thing for 10 or 15 years now, I still haven’t quite cracked the formula for what kinds of PM is able to know when to stop polishing and know when to go for a generational leap. That’s that’s a really challenging, challenging problem for most PMs. But the idea that we can just test our way out of every problem is dangerous because it can really hold a product back.