Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Yulin mistral #26

Open
wants to merge 28 commits into
base: main
Choose a base branch
from
Open

Yulin mistral #26

wants to merge 28 commits into from

Conversation

yulinw2
Copy link

@yulinw2 yulinw2 commented Feb 21, 2025

Add Mistral Model based on main branch.

  1. Supporting Sliding window by introducing a sliding KV cache (Can test it in generate.py by using sliding_window and sliding_size)
  2. Supporting GQA (use flashinfer or self impelemented gqa function in cache.py, both can work)

dreaming-panda and others added 28 commits January 13, 2025 20:20
@dreaming-panda
Merge pull request Infini-AI-Lab#1 from Infini-AI-Lab/main
721da76 
readme
@eltociear
chore: update question.jsonl
9f42c32 
inital -> initial
@dreaming-panda
Merge pull request Infini-AI-Lab#11 from Infini-AI-Lab/main
d34599e 
Qwen Support
@dreaming-panda
Merge pull request Infini-AI-Lab#9 from eltociear/patch-2
b0df344 
chore: update question.jsonl
@kushaann @dreaming-panda
Updated reqiurements.txt transformers -> 4.48 (includes gemma updates
8357238 
that match llama format)
Add gemma.py, gemma_layer.py to models/* for gemma2 support
Update model_utils to support gemma-specific norm and attention
calculation
Add model registration to auto_models
Add prompt formats for instruction tuned and regular gemma2
@kushaann @dreaming-panda
Clean up comments
4dfe551
@dreaming-panda
Merge pull request Infini-AI-Lab#14 from kushaann/v0.1.0
b78a536 
Gemma2 Support
@dreaming-panda
mistral
5b20ff6
@dreaming-panda
mistral
a8da899
@dreaming-panda
Merge branch 'v0.1.0' of https://github.com/Infini-AI-Lab/UMbreLLa in…
9150344 
…to v0.1.0
@dreaming-panda
ministral
5c7f080
@dreaming-panda
gemma
2df6074
@dreaming-panda
draft
bd11c54
@dreaming-panda
draft
f8116e7
@dreaming-panda
draft
7dcbf0d
@dreaming-panda
draft
8b7c7ea
@dreaming-panda
draft
a872e02
@dreaming-panda
draft
1c14661
@dreaming-panda
draft
71aff1b
@dreaming-panda
draft
a706ae1
@dreaming-panda
draft
835dabd
@dreaming-panda
draft
ff1ac17
@yulinw2
mistral 24b
427a318
@yulinw2
debug
aae6d83
@yulinw2
debug
88322f5
@yulinw2
debug
25aa294
@yulinw2
delete unncessary files
5fe65f3
@yulinw2
delete unncessary files
75cd347
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@yulinw2 @dreaming-panda @eltociear @kushaann