I'm trying to configure my poetry pyproject.toml in a way that we can choose between a cpu and a gpu version of the dependencies. The reason for that is that the library torch needs to be installed by specifying the hardware you want to use (see poetry's Instructions for installing PyTorch). My final goal is to be able to install the dependencies on my local computer (Mac with M1 ship) as well as to be able to generate a wheel to send to some distant server and then pip install it (I'm actually using Databricks dbx to do that). Here is the configuration file:
[tool.poetry]
name = "my_project"
version = "0.1.0"
description = ""
authors = ["Me"]
readme = "README.md"
packages = [
{include = "my_project"},
{include = "scripts"}
]
[tool.poetry.dependencies]
python = "3.8.10"
torch = [
{url = "https://download.pytorch.org/whl/cpu/torch-1.12.1-cp38-none-macosx_11_0_arm64.whl", markers="extra == 'cpu'"},
{url = "https://download.pytorch.org/whl/cu102/torch-1.12.1%2Bcu102-cp38-cp38-linux_x86_64.whl", markers="extra == 'gpu'"}
]
[tool.poetry.extras]
cpu=["torch"]
gpu=["torch"]
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
When I run:
poetry install --extras="cpu"
I get the following error:
Because my_project depends on both torch (1.12.1) @ https://download.pytorch.org/whl/cpu/torch-1.12.1-cp38-none-macosx_11_0_arm64.whl and torch (1.12.1+cu102) @ https://download.pytorch.org/whl/cu102/torch-1.12.1%2Bcu102-cp38-cp38-linux_x86_64.whl, version solving failed.