Add image text to text pipeline (#34170)

[frances720]

• Standardize image-text-to-text-models-output

add post_process_image_text_to_text to chameleon and cleanup

Fix legacy kwarg behavior and deprecation warning

add post_process_image_text_to_text to qwen2_vl and llava_onevision

Add post_process_image_text_to_text to idefics3, mllama, pixtral processor

• nit var name post_process_image_text_to_text udop

• nit fix deprecation warnings

• Add image-text-to-text pipeline

• add support for image url in chat template for pipeline

• Reformat to be fully compatible with chat templates

• Add tests chat template

• Fix imports and tests

• Add pipeline tag

• change logic handling of single prompt ans multiple images

• add pipeline mapping to models

• fix batched inference

• fix tests

• Add manual batching for preprocessing

• Fix outputs with nested images

• Add support for all common processing kwargs

• Add default padding when multiple text inputs (batch size>1)

• nit change version deprecation warning

• Add support for text only inference

• add chat_template warnings

• Add pipeline tests and add copied from post process function

• Fix batched pipeline tests

• nit

• Fix pipeline tests blip2

• remove unnecessary max_new_tokens

• revert processing kosmos2 and remove unnecessary max_new_tokens

• fix pipeline tests idefics

• Force try loading processor if pipeline supports it

• revert load_processor change

• hardcode loading only processor

• remove unnecessary try except

• skip imagetexttotext tests for kosmos2 as tiny model causes problems

• Make code clearer

• Address review comments

• remove preprocessing logic from pipeline

• fix fuyu

• add BC resize fuyu

• Move post_process_image_text_to_text to ProcessorMixin

• add guard in post_process

• fix zero shot object detection pipeline

• add support for generator input in pipeline

• nit

• change default image-text-to-text model to llava onevision

• fix owlv2 size dict

• Change legacy deprecation warning to only show when True

[promptless]

📝 Documentation updates detected!

Promptless has analyzed your changes and created a documentation update PR. You can review the proposed documentation changes here:
#5

Please review the suggested updates to ensure they accurately reflect your changes.