Help: How do I fine-tune GPT-4 on honeycomb images to predict colony health?

I’ve been training a GPT‑4 model on a dataset of honeycomb images and colony health metrics. The results are promising, but I’m stuck at the evaluation stage. Also, I’m looking for a position in an apiary tech startup or a research lab that blends AI with apiculture.

Here’s what I did:

• Collected 12,000 labeled images of honeycomb cells, annotated with health indicators (cancerous, varroa infestation, brood pattern anomalies).
• Preprocessed with ResNet50 as a feature extractor, then fed into a GPT‑4 encoder‑decoder architecture for sequence‑to‑sequence predictions of health scores.
• Used cross‑entropy loss and a custom F1 macro metric for multi‑label classification.
• Fine‑tuned on 80% of the data, validated on 10%, tested on 10%.

Problems:

• Overfitting: training loss drops to 0.02 but validation stays at 0.35.
• Attention maps are noisy; unclear which cells the model focuses on.
• I tried a Vision Transformer (ViT) backbone, but it didn’t improve performance.

What I need:

• Advice on regularization techniques (dropout, weight decay) that work with transformer‑based models.
• Tips for visualizing attention over honeycomb cells.
• Any job openings or research projects that need AI expertise in beekeeping.

If you can help, I’ll gladly share my code or dataset. Also, if you know someone hiring for a data scientist in apiary tech, let me know.