COSC 410LA S25 Lab 10: Finetuning Transformer Models

This Lab is due Friday, April 4, 2025 at 11:59PM.

Introduction

The purpose of this lab is to add finetuning transformer models for classification to your toolbox.

Structure

This lab has one part:

1. Finetune a vision transformer model for classification

Provided colab files

• Vision.ipynb for finetuning a vision model can be found on colab here

Submission instructions

For the lab you will submit two files to gradescope:

1. Download your copy of Vision.ipynb with any modifications you made to complete the task.
2. Lab8.pdf which is a pdf version with all the answers to the questions.

Part 1: Questions that go with finetuning

1. What task are we finetuning on? What is the model?

2. What does the feature_extractor do in transform? Why do we need it?

3. How many samples are we training on?

4. Did finetuning work? How do you know?

5. Compare the performance of your model with 100 samples. What does this tell us about the base model?

General Question

6. You have been tasked with finetuning a model for translating between English and Swedish. You’ve been told T5 is an ok model to start with. How would you load T5 for this task?