Facebook open-sources PyText NLP framework

Facebook AI Research is open-sourcing some of the conversational AI tech it is using to power its Portal video chat display and M suggestions on Facebook Messenger.

The company announced today that its PyTorch-based PyText NLP framework is now available to developers.

Natural language processing deals with how systems parse human language and are able to make decisions and derive insights. The PyText framework, which the company sees as a conduit for AI researchers to move more quickly between experimentation and deployment, will be particularly useful for tasks like document classification, sequence tagging, semantic parsing and multitask modeling, among others, Facebook says.

The company has built the framework to fit pretty seamlessly into research and production workflows with an emphasis on robustness and low-latency to meet the company’s real-time NLP needs. The product is responsible for models powering more than a billion daily predictions at Facebook.

Another big highlight is the framework’s modularity, allowing it to not only create new pipelines from scratch but to modify existing workflows. PyText connects to the ONNX and Caffe2 frameworks. It also supports training multiple models at once in addition to distributed training to train models over several runs.

The company obviously isn’t done with making improvements to its NLP frameworks. Facebook says that going forward they’re paying particular attention to working to build an end-to-end workflow for models running on mobile devices.

PyText is available on GitHub.