The enterprise is bullish on AI systems that can understand and generate text, known as language models. According to a survey by John Snow Labs, 60% of tech leaders’ budgets for AI language technologies increased by at least 10% in 2020. And one vendor, OpenAI, says that its premiere language model, GPT-3, is being used by tens of thousands of developers.
Eager for a slice of the pie, new providers have materialized in recent years claiming to bring unique language modeling capabilities to the table. Beyond well-resourced startups like OpenAI, Cohere and Hugging Face, there’s a crop of vendors building services on top of open source AI models. Sitting somewhere in the middle is AI21 Labs, an Israeli company that developed a model — Jurassic-1 Jumbo, which is roughly the size of GPT-3 — and slowly built products around it, including an “AI-as-a-service” platform called AI21 Studio that lets customers create virtual assistants, chatbots, content moderation tools and more.
Investors sense an opportunity, evidently. Today, AI21 Labs closed a $64 million Series B round that values the company at $664 million. Led by Ahren Innovation Capital Fund with participation from Mobileye CEO and co-founder Amnon Shashua, Walden Catalyst, Pitango, TPY Capital and Mark Leslie, the tranche brings A21Labs’ total capital raised to $118.5 million.
Co-founder and CEO Ori Goshen said that the new money will be put toward R&D, particularly developing larger and more sophisticated language models, and recruiting talent. AI21 Labs currently has 120 employees and plans to hire around 50 more by the end of the year, defying the macroeconomic trend.
“Fortunately, the pandemic has positively impacted business — as more companies migrated to remote work, individuals needed to convey in written text what they would normally share verbally,” Goshen told TechCrunch in an email interview. “[Our] proprietary large language models’ core capabilities allow for the ingestion of massive amounts of corporate data use to do … custom content creation, summarization, and classification.”
AI21 Labs was co-founded in 2017 by Goshen, Shashua, and Stanford University professor Yoav Shoham. The company’s first product was Wordtune, an AI-powered writing aid meant to compete with Grammarly, which suggests rephrasing text wherever users type. AI21 Studio was released last August, along with a “pay-as-you-go” service that allows developers to apply for access to custom models fine-tuned on datasets unique to their requirements.
AI21 Labs offers a range of tuning parameters to customize the output of its models. Image Credits: AI21 Labs
Within AI21 Studio, AI21 Labs’ Jurassic-1 family of models can be used for paraphrasing (like generating short product names from product description), extracting figures from text and labeling emails and notes by topic or category. The models can also summarize content through a feature in Wordtune dubbed Wordtune Read, including snippets from articles, reports and PDF files.
Because they’re trained on large amounts of data from the internet, including social media, language models are capable of generating toxic and biased text based on similar language that they encountered during training. AI21 Labs’ models are no different; in early testing, one researcher was able to prompt them to say “people who love Jews are closed-minded.” While AI21 Labs requires customers to agree to a terms of use policy and usage guidelines, it hasn’t implemented filters for potentially toxic content generated by its APIs.
AI21 Labs, which says it manually reviews requests for fine-tuned models to combat abuse, has claimed that its models are “marginally less biased” than GPT-3.
Regardless, according to Goshen, the models have an advantage in that they’re augmented with external knowledge sources like Wikipedia. The latest version of AI21 Labs’ Jurassic-1 model, Jurassic-X, uses what Goshen calls a “modular reasoning knowledge system” to enhance its answers with “discrete reasoning experts” such as online calculators and currency converters. Jurassic-X can answer “nontrivial” math operations phrased in natural language as a result, Goshen says, as well as simplify “complex” questions that might trip up other language models.
Of course, it’s worth noting that AI21 Labs hasn’t commissioned a comparison of its Jurassic-X models with other commercial language models, so claims are all we have to go on.
The company’s questionable recent marketing stunt doesn’t instill enormous confidence. In June, AI21 Labs launched a chatbot modeled on the legal opinions of the late Supreme Court justice Ruth Bader Ginsburg that several AI technology experts characterized as misleading. Responding to the criticism, AI21 Labs said that the chatbot was “just an experiment” and admitted it can give inaccurate responses that should be taken “with a grain of salt.”
When asked, Goshen declined to disclose firm revenue figures or even estimates of growth. But he said that Studio has “hundreds” of paying clients and design partners — none of which he was willing to identify by name — in addition to over 10,000 users of its free plan, while Wordtune has “millions” of users.
Given the cost of training sophisticated models, there’s likely significant investor pressure to expand. AI21 Labs’ own research pegs expenses for developing a text-generating model with 1.5 billion parameters (i.e., variables that the model uses to generate and analyze text) at as much as $1.6 million. Jurassic-1 Jumbo contains 178 parameters. That’s not accounting for hosting costs to serve the models; AI21 Labs says it retains the services of “several” third-party cloud providers both in the U.S. and abroad.
“[There’s a lack] of market knowledge because the language model technology is so nascent and just starting to gain adoption,” Goshen said. “With the new funding, AI21 Labs will continue in its mission of building AI systems with an unprecedented capacity to understand and generate natural language.”