module 'fused_dense_lib' has no attribute 'linear_act_forward'

[tanshuai0219]

I run the code on a A100 GPU but get error:
module 'fused_dense_lib' has no attribute 'linear_act_forward'

[tanshuai0219]

I run the code on a A100 GPU but get error: module 'fused_dense_lib' has no attribute 'linear_act_forward'

my environment：

`Package Version

---

absl-py 2.0.0
accelerate 0.21.0
adabench 1.2.64
aii-pypai 0.1.40.45
aiofiles 23.2.1
aiohttp 3.9.1
aiosignal 1.3.1
aistudio-analyzer 0.0.4.102
aistudio-common 0.0.28.48
aistudio-notebook 2.0.125
aistudio-serving 0.0.0.62
alipay-pcache 0.1.6
aliyun-python-sdk-core 2.14.0
aliyun-python-sdk-kms 2.16.2
altair 5.2.0
annotated-types 0.6.0
ant-couler 0.0.1rc17
antlr4-python3-runtime 4.9.3
anyio 4.2.0
apex 0.1
archspec 0.2.1
argo-workflows 3.5.1
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
astroid 3.0.2
asttokens 2.4.1
async-timeout 4.0.3
atorch 1.1.0rc8
attrs 23.1.0
autopep8 2.0.4
av 11.0.0
backcall 0.2.0
beautifulsoup4 4.12.2
bigmodelvis 0.0.1
bitarray 2.8.5
bitsandbytes 0.39.0
bleach 6.1.0
blinker 1.7.0
boltons 23.0.0
boto3 1.34.2
botocore 1.34.2
Brotli 1.0.9
cachetools 3.1.1
cattrs 23.2.3
certifi 2023.11.17
cffi 1.16.0
charset-normalizer 2.0.4
cheroot 10.0.0
click 6.7
click-config-file 0.6.0
cloudpickle 3.0.0
colorama 0.4.6
comm 0.2.1
conda 23.11.0
conda-content-trust 0.2.0
conda-libmamba-solver 23.12.0
conda-package-handling 2.2.0
conda_package_streaming 0.9.0
configobj 5.0.8
configparser 6.0.0
contourpy 1.1.1
couler-core 0.1.1rc11
crcmod 1.7
cryptography 41.0.7
cycler 0.12.1
Cython 3.0.6
datasets 2.15.0
debugpy 1.8.0
decorator 5.1.1
deepspeed 0.10.3
defusedxml 0.7.1
delta-center-client 0.0.4
Deprecated 1.2.14
deprecation 2.1.0
diffusers 0.24.0
dill 0.3.7
distlib 0.3.8
distro 1.8.0
dlrover 0.3.6
docker 4.1.0
docstring-to-markdown 0.13
easydl-sdk 0.0.6
einops 0.7.0
entrypoints 0.4
evaluate 0.4.0
exceptiongroup 1.2.0
executing 2.0.1
fairscale 0.4.1
fastapi 0.108.0
fastjsonschema 2.19.1
fastmoe 1.0.0
fasttext 0.9.2
fe 0.3.33
ffmpeg-python 0.2.0
ffmpy 0.3.1
filelock 3.13.1
flake8 6.1.0
flash-attn 2.0.4
flash-attn-1 0.2.6.post2
Flask 3.0.0
flatbuffers 24.3.25
fonttools 4.46.0
fqdn 1.5.1
frozenlist 1.4.1
fsspec 2023.10.0
ftfy 6.1.3
future 1.0.0
gitdb 4.0.11
GitPython 3.1.40
google-auth 2.25.2
google-auth-oauthlib 0.4.6
gradio 4.13.0
gradio_client 0.8.0
grpcio 1.34.1
grpcio-tools 1.34.1
h11 0.14.0
hjson 3.1.0
httpcore 1.0.2
httpx 0.26.0
huggingface-hub 0.19.4
icetk 0.0.7
idna 3.4
importlib-metadata 7.0.0
importlib-resources 6.1.1
iniconfig 2.0.0
ipykernel 6.28.0
ipython 8.12.3
ipython-genutils 0.2.0
isodate 0.6.1
isoduration 20.11.0
isort 5.13.2
itsdangerous 2.1.2
jaraco.functools 4.0.0
jax 0.4.13
jaxlib 0.4.13
jedi 0.19.1
jedi-language-server 0.41.2
Jinja2 2.11.3
jinjasql 0.1.8
jmespath 0.10.0
joblib 1.3.2
jsonpatch 1.32
jsonpath-ng 1.6.0
jsonpointer 2.1
jsonschema 4.20.0
jsonschema-specifications 2023.11.2
jupyter_client 8.6.0
jupyter_core 5.7.1
jupyter-events 0.9.0
jupyter-lsp 2.2.1
jupyter_server 2.10.1
jupyter_server_terminals 0.5.1
jupyterlab_pygments 0.3.0
kiwisolver 1.4.5
kmitool 0.0.9
kubemaker 0.2.17
kubernetes 9.0.0
langdetect 1.0.9
libmambapy 1.5.3
loralib 0.1.1
lsh 0.1.2
lsprotocol 2023.0.0
lxml 4.9.3
M2Crypto 0.38.0
Markdown 3.5.1
markdown-it-py 3.0.0
MarkupSafe 2.0.1
marshmallow 3.20.1
matplotlib 3.7.4
matplotlib-inline 0.1.6
mccabe 0.7.0
mdurl 0.1.2
mediapipe 0.10.11
megatron.core 0.1
menuinst 2.0.1
mistune 0.8.4
ml-dtypes 0.2.0
mock 5.1.0
more-itertools 10.1.0
mpi4py 3.1.5
mpmath 1.3.0
msgpack 1.0.7
multidict 6.0.4
multiprocess 0.70.15
nbclient 0.5.13
nbconvert 6.4.4
nbformat 5.9.2
nest-asyncio 1.5.8
networkx 3.0
ninja 1.11.1.1
nltk 3.8.1
notebook 6.4.6
numpy 1.23.5
oauthlib 3.2.2
odps 3.5.1
omegaconf 2.2.3
opencv-contrib-python 4.9.0.80
opencv-python 4.8.1.78
opendelta 0.3.2
opt-einsum 3.3.0
orjson 3.9.10
oss2 2.6.0
osscmd 0.4.5
overrides 3.1.0
packaging 23.1
pandas 1.0.0
pandocfilters 1.5.0
parameterized 0.9.0
parso 0.8.3
pathos 0.3.0
peft 0.3.0
peppercorn 0.6
pexpect 4.9.0
pickleshare 0.7.5
Pillow 9.3.0
pip 23.3.1
pkgutil_resolve_name 1.3.10
platformdirs 3.10.0
pluggy 1.0.0
ply 3.11
pox 0.3.3
ppft 1.7.6.7
prettytable 3.9.0
prometheus-client 0.19.0
prompt-toolkit 3.0.43
protobuf 3.20.0
psutil 5.9.6
PTable 0.9.2
ptyprocess 0.7.0
pure-eval 0.2.2
py 1.11.0
py-cpuinfo 9.0.0
py-spy 0.3.14
pyaml 21.10.1
pyarrow 12.0.0
pyarrow-hotfix 0.6
pyasn1 0.5.1
pyasn1-modules 0.3.0
pybind11 2.11.1
pycodestyle 2.11.1
pycosat 0.6.6
pycparser 2.21
pycryptodome 3.19.0
pydantic 1.10.8
pydantic_core 2.14.6
pyDes 2.0.1
pydocstyle 6.3.0
pydub 0.25.1
pyflakes 3.1.0
pygls 1.2.1
Pygments 2.17.2
pyhocon 0.3.60
pyinotify 0.9.6
pylint 3.0.3
pynvml 11.4.1
pyodps 0.11.4.1
Pyomo 6.7.0
pyOpenSSL 23.2.0
pyparsing 3.1.1
PySocks 1.7.1
pytest 7.4.3
python-dateutil 2.8.2
python-json-logger 2.0.7
python-lsp-jsonrpc 1.1.2
python-lsp-server 1.9.0
python-multipart 0.0.6
pytoolconfig 1.2.6
pytz 2023.3.post1
PyWavelets 1.4.1
PyYAML 6.0.1
pyzmq 25.1.2
ray 2.9.0
referencing 0.32.0
regex 2023.10.3
requests 2.31.0
requests-file 1.5.1
requests-oauthlib 1.3.1
requests-toolbelt 1.0.0
responses 0.18.0
retry 0.9.2
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rich 13.7.0
rope 1.11.0
rouge-chinese 1.0.3
rouge-score 0.1.2
rpds-py 0.14.1
rsa 4.9
ruamel.yaml 0.16.10
ruamel.yaml.clib 0.2.6
ruff 0.1.11
ruff-lsp 0.0.49
s3transfer 0.9.0
safetensors 0.4.1
scikit-learn 1.3.2
scipy 1.10.1
semantic-version 2.10.0
Send2Trash 1.8.2
sentencepiece 0.1.97
setuptools 68.2.2
shellingham 1.5.4
six 1.16.0
smmap 5.0.1
sniffio 1.3.0
snowballstemmer 2.2.0
sounddevice 0.4.6
soupsieve 2.5
sqlparse 0.4.4
stack-data 0.6.3
starlette 0.32.0.post1
stringcase 1.2.0
StringGenerator 0.4.4
sympy 1.12
tabulate 0.8.2
tensorboard 2.11.0
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorboardX 2.6
termcolor 2.4.0
terminado 0.18.0
testpath 0.6.0
threadpoolctl 3.2.0
tinycss2 1.2.1
titans 0.0.7
tldextract 5.1.1
tokenizers 0.13.3
tomli 2.0.1
tomlkit 0.12.0
toolz 0.12.0
torch 2.1.0+cu121
torchaudio 2.1.0+cu121
torchpippy 0.1.1+cecc4fc
torchvision 0.16.0+cu121
tornado 6.4
tqdm 4.65.0
traitlets 5.14.1
transformers 4.31.0
triton 2.1.0
typer 0.9.0
types-python-dateutil 2.8.19.20240106
typing_extensions 4.9.0
tzdata 2023.3
ujson 5.9.0
uncertainty-calibration 0.1.4
Unidecode 1.3.7
unifile-sdk 0.1.14
uri-template 1.3.0
urllib3 1.26.18
uvicorn 0.25.0
virtualenv 20.25.0
watchdog 2.3.1
wcwidth 0.2.12
web.py 0.62
webcolors 1.13
webencodings 0.5.1
websocket-client 1.7.0
websockets 11.0.3
Werkzeug 3.0.1
wfbuilder 1.0.56.43
wget 3.2
whatthepatch 1.0.5
wheel 0.41.2
wrapt 1.16.0
xattr 1.0.0
xxhash 3.4.1
yacs 0.1.8
yapf 0.40.2
yarl 1.9.4
zdfs-dfs 2.3.2
zeep 4.2.1
zipp 3.17.0
zstandard 0.19.0`

[keyu-tian]

Try from flash_attn.ops.fused_dense import fused_mlp_func and check if your fused_mlp_func works.
Maybe you've installed flash attention but haven't compiled that fused mlp operator. You may run:

cd /path/to/flash-attention
cd csrc/fused_dense_lib && pip3 install .

to compile and install it.

[tanshuai0219]

Try from flash_attn.ops.fused_dense import fused_mlp_func and check if your fused_mlp_func works. Maybe you've installed flash attention but haven't compiled that fused mlp operator. You may run:

cd /path/to/flash-attention
cd csrc/fused_dense_lib && pip3 install .

to compile and install it.

Thanks for your quick reply, and I get the results~