Palantir – Masters of Big Data
Palantir has developed massive expertise as a big data "consultancy", showing its clients the value that can be unlocked in their own data.
Palantir is in many ways the epitome of the power of big data and today’s enterprise needs to better leverage their own data. At its core, Palantir is a big data consultancy: a company that deploys teams of engineers and business people to client companies to help them set up more powerful data visualization and analytical infrastructure. However, through another lens, Palantir can be seen as an outsourced solution for other companies to capture more value out of their own data.
Palantir was spearheaded in the wake of 9/11 by a team of Silicon Valley heavyweights including Peter Thiel, largely in response to the government’s need for greater insights into data for counter-terrorism goals. Soon, Palantir began to serve myriad other government institutions for their various big data needs, a notable example being data-driven anti-fraud projects.
Using their government projects to build infrastructure, expertise, and credibility, Palantir soon began to expand to commercial projects. Today, its clients come from all industries, notable ones including finance (anti-fraud being a great concern for insurance and bank companies), law (where legal research projects are becoming more complex and data-intensive), and pharma (where R&D is similarly becoming more complex). Despite growing quickly over the last decade and being valued at over $15B, Palantir reportedly is not going to seek an IPO.
Palantir’s value creation stems from its ability to create big data solutions for other companies. In the world of increasing sources and complexity of data, there is value to be unlocked in the insights that data can provide (which exist, but are hard to get to), and value to be captured by a company that can help other companies unlock those insights. Palantir’s consulting services involve assessing the types of data the company already generates, combining it with publicly available sources of information, and creating more robust and flexible database structures and visualization tools to allow other company functions to better perform their jobs.
For example, one Palantir project has helped pharma companies detect fraudulent prescription claims. Palantir can combine proprietary data on drug claims from individual pharmacies with all manners of publicly available data, such as business information and pharmacy addresses. It then creates a portal where an analyst can see, for example, a map of all pharmacies within an area, any overlapping names of people who have submitted suspicious prescription claims, and then a timeline of when all these claims happened. Using Palantir’s tools, the client company can ostensibly better detect fraud – value creation in the form of a cost reduction. Palantir also captures some of this value in the form of its fees.
A couple things are noteworthy about this process and operating model. First, Palantir is not necessarily in the business of providing the ultimate solution for its clients – for example, it creates the systems to help the company better detect fraud, but it does not always draw conclusions itself about which accounts are fraudulent. This allows Palantir to stay within its own realm of expertise (i.e. creating big data systems) without having to develop numerous, deep domain-specific areas of knowledge (e.g. detecting pharmacy-related fraud).
Second, Palantir is not really in the business of helping the company develop completely new sources of data. There is already much value to be created by simply leveraging data the client already collects with publicly available data.
Third, by slightly abstracting away from the specifics of the industry or client, Palantir develops solutions that are replicable across clients. Principles of visualization and working with private and public data apply beyond the boundaries of a given firm, and Palantir is well-positioned to develop this area of cross-firm and cross-industry expertise.
Fourth is a critical aspect of the process that is not as well advertised in public materials: Palantir generally only accepts projects where there is strong support from the highest executive level, often the CEO. Its projects could be seen as a threat to the company’s internal IT or business intelligence units, so it can avoid some politicking by demanding a mandate from the very top.
Finally, a critical key to the Palantir operating model is its talent – it has a high bar for recruiting engineers (who today would probably rather join a “sexy” Silicon Valley company than the IT department of most of Palantir’s clients) and also those with business backgrounds (who can better deal with client relations). Much like management consulting firms, Palantir uses human capital to create value for a range of client businesses, human capital that its clients largely cannot attract. In some senses then, Palantir combines elements of work traditionally done by IT or infrastructure consultancies (e.g. SAP, Salesforce, Oracle) with the high-level mandate and broad human capital of management consultancies (e.g. McKinsey, Bain, BCG).
Palantir is an impressive manifestation of the power of data and new kinds of businesses that can be built on top of data expertise.
With pretty universal agreement that the ability to use and analyze data is going to be important to future business, how long do you think businesses will continue to outsource these types of projects to Palantir vs. starting to build this expertise internally? If they continue to rely on a 3rd party, businesses might find themselves “left-behind” and less competitive than they need to be to be successful going forward.
I would guess that it will be a long time before businesses start to build this kind of expertise internally – it seems like it’d take a lot of special human capital to do this.
If anything, I think the biggest “threat” to Palantir is that eventually there are enough open-source mini-solutions that, when cobbled together, would allow a very small number of engineers to build up the same kind of data infrastructure that Palantir can provide.
But in the meantime, you’re right – businesses that don’t harness their data will probably be slowly “left behind”. And that’s great news for Palantir, because it’ll help attract more customers 🙂
Great blog!! Thanks Allen! What I really like about this company is that they seem to focus on the one thing they are good at! And: they only accept project where they have CEO backing! When I compare that with the big consulting firms: I think they do have the knowledge and ability to do that stuff, meaning they have employees that know how to do it. The problem is, the have even more employees that do not know how to do it and their struggle is, to prevent the employees that have no clue what to do, from still working on this projects, instead of just getting the experts in. This is the challenge of larger consulting firms with a huge variety in services – the internal political battles over clients and the sheer size which makes it really difficult sometimes to get the right people on the right topic at the right client!
This won´t change I guess so quickly, why I think that Palantir has quite a solid edge in the business…
There’s not doubt this is an impressive company with an incredible valuation ($10m per employee) but I have a couple of concerns about their model.
First, it’s clear that introducing new systems in a business does not alone drive business value. Once the systems are introduced it is in the hands of the clients to prove the new tools are working. If companies systematically lack capability to generate value from the systems introduced by Palantir this will put the value of Palantir’s services questions and may stand in the way of repeat customers.
Second, Palantir’s valuation is a clear indication investors expect the company to grow rapidly. This puts into question both Palantir’s requirement for CEO backing of projects as well as its ability to attract top engineering talent, especially if some of the excitement of new solutions gives way to the more mundane work of porting previous solutions to new clients.
Great post, and very cool company. I’m particularly interested in the points you make about Palantir not drawing conclusions itself based on the data it crunches for its clients. On the one hand, this makes perfect sense. Why take on the additional risk associated with the actions companies take after they see their data in a new light? On the other hand, it’s possible that Palantir is missing out on the potential upside of the decisions it’s helping companies make. If a company saves $1B from a decision made using a Palantir tool, should Palantir be satisfied with the flat fee they charged in the beginning? I suppose if Palantir can continue to grow (as Vlad points out), then taking fees rather than a share of cost savings, etc. may be totally satisfactory and keep the risk in the right place. But maybe Palantir can develop deeper expertise in particular disciplines/industries, and snag a bigger chunk of the pie.
Allen do you know if Palantir has any emerging competitors that pose a serious threat? I have always been fascinated by Palantir and their ability to win huge contracts early on in the life of the company, and I have wondered how a competitor would go about getting a foot in the door at this point. In general, it seems like an extremely lucrative space that is far less crowded than other hyped up buzzwords in the Valley right now…Palantir is very willing to not attract attention to itself (hence no IPO) while quietly dominating the big government contract data mining / intelligence markets.
Good question – I don’t know of a specific threat. But, I would think that various parts of their service offering are being threatened by substitutes. For example, you might be able to cobble together something that looks like a Palantir solution using a combination of AWS for infrastructure + various open-source libraries or even smaller SaaS companies’ services. As commenter JG alluded to above, perhaps over time this means the cost of developing a Palantir-like solution internally will go down (but I still think you need hard-to-find and harder-to-attract engineering talent to do it), and Palantir will need to move “up-market”. Or, perhaps in time somebody like AWS will swoop in and provide a threatening end-to-end alternative to Palantir?
Allen, great post!
I read that Palantir partners with law enforcement agencies to help to fight crime. They equip officers and agencies with tools that help to analyze intelligence, produce reports on crime and provide data for investigations. Some speculate that one of Palantir’s projects is to help governments to predict terror attacks. Governments use companies like Palantir to help them to analyze what is the risk of terror attack and how it is changing based on system that is tracking events that already happened. I’m not sure how much of it is true, because Palantir was very secretive about its product.