Looks good so far! The TODO list from here is:

• Make the transformers imports relative imports so that we stop getting circular import errors in the CI
• Add the phi3_5.md doc (you can copy the layout from a similar model and copy the text from the model card)
• pip install transformers[quality] and make fixup or make style to get the repo consistency checks green

Once everything is green, then:

• Add tests (you can copy them from another similar VLM and just change model names etc.)

Once the new tests are green as well, then:

• Convert the modeling and configuration files to modular, then regenerating the modeling/config files and confirm everything passes

At that point, the PR should be finished!

Fixes #36166!

I've run into an issue when trying to run the model. From Modeling_phi3_v.py I'm getting "AttributeError: 'DynamicCache' object has no attribute 'get_max_length'" This might be due to a change to the DynamicCache class, but I really can't tell!

@Dahlbomii hey! Yes, we deprecated get_max_length from cache a few versions ago, now it has to be Cache.get_max_cache_shape(layer_idx)

Woah - this means the remote code version of Phi-3.5-vision-instruct is broken on main right now and requires a fixed older version of transformers. Definitely makes this PR more urgent/important!

cc @zucchini-nlp @gante I can't figure this one out either! The test_generate_continue_from_inputs_embeds test is failing for Phi-3V but I don't really understand how generate() handles inputs_embeds - when I added some breakpoints it seems to generate an input_ids tensor with shape (batch_size, 0), which then causes errors in Phi.

Seems like Phi-3.5V was not updated for the latest changes we had in generation, probably it has its own custom generation loop. For this test, we can call super().prepare_input_for_generation(kwargs) to get all the inputs processed correctly. Then image related kwargs can be set to None if needed. You can take a look on how we did it in other VLMs, hope it solves the issue

@zucchini-nlp yes, that was it, sorry! They don't override generate() but they do override prepare_input_for_generation() and discard inputs_embeds after the first step.

@Rocketknight1 for reasons that escape me, when the test processor tries to call self.get_component for the tokenizer, it breaks. Any insight as to why?