Type Enhancement for Func Transforms and Bug Fix

[XXXXRT666]

Proposed changes

1. Functions lose precise type hints after transformation. For example, mx.compile only annotates its input and output as Callable, so the original function signature and return types are not preserved in the type system.
2. Errors occur during the stub gen.

Type Hints After This PR:

Checklist

Put an x in the boxes that apply.

• I have read the CONTRIBUTING document
• I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
• I have added tests that prove my fix is effective or that my feature works
• I have updated the necessary documentation (if needed)

[XXXXRT666]

Also, python setup.py generate_stubs no longer works, despite being mentioned here.

[zcbenz]

Also, python setup.py generate_stubs no longer works, despite being mentioned here.

Thanks for noticing this, the stubs generation is now part of standard build command after #2949. Can you remove that in document?

[zcbenz]

I'm not sure if this actually fixes the problem you saw, the exact path has already been passed in the stubgen target:

mlx/python/src/CMakeLists.txt

Lines 35 to 40 in d2bef3c

	# Run stubgen -m mlx.core -i python -p _stub_patterns.txt -o python/mlx
	RECURSIVE
	MODULE
	"mlx.core"
	PYTHON_PATH
	"${CMAKE_CURRENT_SOURCE_DIR}/.."

Can you share the exact command that you used?

[XXXXRT666]

the command is pip install git+https://github.com/ml-explore/mlx.git -vv and pip install -vv git+https://github.com/XXXXRT666/mlx.git@type-fix

[XXXXRT666]

CMAKE_CURRENT_SOURCE_DIR = ~/mlx/python/src in my environment, but mlx.core is generated in ~/mlx/build/lib.macosx-11.0-arm64-cpython-310/mlx/core.cpython-310-darwin.so

[zcbenz]

Thanks a lot for the information! Can you check if #3009 fixes the error for you?

[XXXXRT666]

Thanks a lot for the information! Can you check if #3009 fixes the error for you?

It doesn't fixed the error, I think the correct path is $<TARGET_FILE_DIR:core>/.. as <TARGET_FILE_DIR:core> is the path to the directory of the target .so file, which is ~/mlx/build/lib.macosx-11.0-arm64-cpython-310/mlx

[zcbenz]

Oh I only tested local install and I guess it searched from the other path, thanks for testing. Do you mind checking the updated PR again? This time it passes the remote install for me. pip install -v git+https://github.com/zcbenz/mlx.git@stubgen-python-path

[XXXXRT666]

Yes no error occurs, stubs are generated correctly

[XXXXRT666]

Thanks for noticing this, the stubs generation is now part of standard build command after #2949. Can you remove that in document?

I didn't found it in the repo, maybe it's has been modified

[zcbenz]

I didn't found it in the repo, maybe it's has been modified

Sorry my bad, it was already removed in #2949, the website has not been updated yet.

[zcbenz]

I don't have much experience with python typing but after reading the docs I think this is just standard usage. Still needs a check from @awni .