Skip to content

Add interface for using generic CoreML LLMs

Abhinav Jain requested to merge compile_cmd_line_generic_model_broken into main

This PR adds an interface for adding any CoreML LLM to the CoreML encoder.

Models can be added by following the steps in adding_new_model.md. Once they are added, they can be accessed through the LLMEmbeddings object or the CoreMLEncoder object using a string identifier.

The PR makes changes to the CoreMLEncoder and LLMEmbeddings (FKA MiniLMEmbeddings) classes to support the use of any model. This is done through the added macros, which generate interfaces for interacting with models and code to validate model string identifiers & configure the LLMEmbeddings class to use the correct model for prediction.

The test in AllMiniLM_sampleTest.swift was enabled as part of this change, in order to validate that the interfaces work with multiple models and to provide an example of how to use the LLMEmbeddings object.

Merge request reports