Text classification using LLMs

Follow the full discussion on Reddit.
Hi, I am looking for a solution to do supervised text classification for 10-20 different classes spread across more than 7000 labelled data instances. I have the data in xlsx and jsonl formats, but can be converted to any format required easily. I've tried the basic machine learning techniques and deep learning also but I think LLMs would give higher accuracy due to the transformer architecture. I was looking into function calling functionality provided by Gemini but it is a bit complicated. Is there any good framework with easy to understand examples that could help me do zero shot, few shot and fine tuned training for any LLM? A Colab session would be appreciated. I have access to Colab pro also if required. Not any other paid service, but can spend upto $5 (USD). This is a personal research project so budget is quite tight. I'd really appreciate if you could direct me to any useful resources for this task. Any LLM is fine.