AAM Spotlight: Sergio Cecutta, SMG Consulting

A founder and partner at SMG Consulting, Sergio Cecutta has emerged as a thought leader in the advanced air mobility space, boosted by the popular AAM Reality Index, a widely referenced ranking of startup OEMs.

But as the popularity of the AAM Reality Index has grown, so too have questions over its methodology. In a recent interview with Aviation Week, Cecutta explains the purpose of the index and the criteria he uses to judge companies. A partial transcript follows:

Aviation Week: What is the purpose of the AAM Reality Index? What does it attempt to measure?

Cecutta: The reason for developing the index was to bring clarity to the progress of each OEM, because there is a lack of standardized milestones in this industry, and that has made allowed some companies to highlight things that maybe do not deserve to be highlighted.

Our basic idea was simply to track the progress of each OEM toward certification and production. The goal was never to predict which company has the better business model or which is going to survive. That’s why a company could be the top company in the index today, but still fail and disappear in the future.

Why not try and evaluate the business model? Wouldn’t that be useful?

My corporate background taught me that my opinion counts for nothing—all that matters is what is backed by data. So what we wanted to do was to leave out anything opinion-based. For me to say what company will survive is pure speculation. In order to track their business case and likelihood to survive in an objective way, you’d need insider information that is not available.

When asked, I’m more than happy to share who I think is going to be successful and who is at risk. But I don’t want to rank companies that way, because I think the value of the index would be significantly diminished if I start inserting my own opinions.

What are the criteria on which you score companies?

There are five parameters: funding, team, technological progress, certification progress and production capability. Each of those parameters has about five to 10 specific milestones that we track. There is always going to be some degree of subjectivity, but our goal here is to be as objective as possible by comparing everyone against very specific milestones.

Let’s start with funding. What exactly are we measuring? Are we just looking at the total amount raised?

I don’t care about the total amount raised. When it comes to funding, I want to know what they’ve accomplished with the funding they’ve received. It’s about how efficiently they spend that cash and we measure that against their progress to date. So it’s not about who received the most money. It’s about, ‘Do you have enough money to fly a prototype? Do you have enough to certify? Do you have enough to produce and at what scale?’

That’s why a company like EHang is ranked very high even though they have not raised nearly as much as Joby or Archer. They don’t need to raise as much because their costs are much lower.

How do you evaluate the capabilities of a company’s team? Is it just about who has the most relevant experience?

We read the resumes of every single member of the leadership team to see if the skills they have match their stage of company development. 

When you’re a startup making a prototype, you need a more engineering-dependent skill set. When you start to commercialize, you need functions like sales, customer support and operations. So we are looking at the expertise and skill sets of the leadership team compared to the skills that the company needs at that point in time.

Lilium is a great example, where Daniel [Wiegand], who led the engineering development as a young founder, has since stepped into a more visionary role and brought on board a lot of people like [CEO] Klause Roewe who can handle the execution. 

Do we look for a CEO skillset when the company is building prototypes? No. Do we look for it when they want start commercializing? Absolutely.

What about technological progress? Do you use the same TRL levels (Technology Readiness Levels) used by NASA and industry?

We need to use a metric that is technology-agnostic. We don’t care if you’re multicopter or lift-plus-cruise or whatever. And with that in mind, what better metric is there than the TRL scale from NASA? This kind of objective evaluation is what it was born to do.

The two biggest milestones for TRL are six and nine. TRL 6 is when you have developed something in the relevant environment, and TRL 9 is when you’re basically done. So, as an example, in the case of an eVTOL [electric vertical-takeoff-and-landing vehicle], we don’t consider TRL 6 until you’ve transitioned. For an eSTOL [electric short-takeoff-and-landing] like Electra, TRL 6 is when they’ve demonstrated their STOL landing and takeoffs work.

Using the TRL scale allows us to compare a hybrid with an eVTOL or eSTOL. We’re not picking winners and losers; we’re purely looking at the technology readiness of whatever the chosen configuration happens to be.

Besides TRL, we also infuse some additional detail to distinguish between subscale, full-scale, manned and unmanned prototypes to provide more specificity.

How about certification?

When it comes to certification, we have specific milestones to follow; DOA, POA, G-1, G-2, G-3, etc. These are mostly black and white and leave very little room for subjectivity. But we want to see the work is actually completed. I don’t care if 70% of the deliverables for your G-1 are submitted. Is it done or not? That’s what we care about. But in short, we are just tracking the same kinds of milestones as the rest of the industry.

How do you measure production capabilities of startup OEMs?

The beauty of production is that NASA also developed an MRL [manufacturing readiness level] scale that goes from one to 10. The MRL just looks at the manufacturability of aircraft and specific components. But in addition to that, we break down scale into different pieces; is it LRIP [low-rate initial production]? Is it production at a large scale? Is it a mass scale of more than 1,000 per year?

We’re really looking at what they have today. Volocopter has a production line that is open and can make up to 50 aircraft per year. Archer soon will have a plant that can produce up to 650, but how much production will it have when it opens? That’s a different story. We’re looking at a snapshot of where they stand today.