I have setup an mlflow service in a VM and I am able to serve the model using mlflow serve command. Wanted to know if we can host multiple models in a single VM ?

I am using the below command to serve a model using mlflow in a vm.

command:

/mlflow models serve -m models:/$Model-Name/$Version --no-conda -p 443 -h 0.0.0.0

Above command creates a model serving and runs it on 443 port. Is it possible to have an endpoint like below being created with model name in it ?

Current URL: https://localhost:443/invocations

Expected URL: https://localhost:443/model-name/invocations ?

CodePudding user response：

I believe that mlflow models serve will only accept POST input to the /invocations path.
If you want something custom I would suggest:

1. Seldon
2. Create a simple Flask app to do it as illustrated in this blog post.

CodePudding user response：

OSS MLflow only serves one model.

While Databricks MLflow model server doesn't yet support first-class multi-model serving, you can use registered model versions to serve different models. Since all model versions are deployed in a Databricks model server, you can reference them as https://mycompany.com/model/me/1/invocations or https://mycompany.com/model/me/2/invocations.