design final pipeline version #25

New issue

Open

opened 2025-11-24 15:23:27 +00:00 by AndreFerreira5 · 0 comments

AndreFerreira5 commented

2025-11-24 15:23:27 +00:00

(Migrated from github.com)

the new pipeline should follow: continuous integration -> continuous development -> continuous staging -> continuous deployment

this new pipeline will be way simpler because we found out that self hosting a github runner in our private server (that hosts the APIs and MLFLow server) is beneficial to us, both in terms of the maximum run time (6h with github runners, 72h with self hosted) and in terms of not needing to setup a VPN in the workflows

additionally, the docker compose file that has both the mlflow and training api container will be separated into two docker compose files, to be able to manage each container individually, instead of together (the training API will be updated much more than the mlflow server, which will practically stay as it is for the rest of the development)

right now we only have a testing action and a deploy action which deploys the training api, inference api and the mlflow server inconditionally, even if they dont change (only because of the commit/push that triggers the workflows) the new pipeline should follow: continuous integration -> continuous development -> continuous staging -> continuous deployment this new pipeline will be way simpler because we found out that self hosting a github runner in our private server (that hosts the APIs and MLFLow server) is beneficial to us, both in terms of the maximum run time (6h with github runners, 72h with self hosted) and in terms of not needing to setup a VPN in the workflows additionally, the docker compose file that has both the mlflow and training api container will be separated into two docker compose files, to be able to manage each container individually, instead of together (the training API will be updated much more than the mlflow server, which will practically stay as it is for the rest of the development)