|
8 | 8 | "<div class=\"align-center\">\n", |
9 | 9 | "<a href=\"https://unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n", |
10 | 10 | "<a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord button.png\" width=\"145\"></a>\n", |
11 | | - "<a href=\"https://docs.unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a></a> Join Discord if you need help + ⭐ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐\n", |
| 11 | + "<a href=\"https://unsloth.ai/docs/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a> Join Discord if you need help + ⭐ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐\n", |
12 | 12 | "</div>\n", |
13 | 13 | "\n", |
14 | | - "To install Unsloth on your own computer, follow the installation instructions on our Github page [here](https://docs.unsloth.ai/get-started/installing-+-updating).\n", |
| 14 | + "To install Unsloth on your own computer, follow the installation instructions on our Github page [here](https://unsloth.ai/docs/get-started/installing-+-updating).\n", |
15 | 15 | "\n", |
16 | 16 | "You will learn how to do [data prep](#Data), how to [train](#Train), how to [run the model](#Inference), & [how to save it](#Save)\n" |
17 | 17 | ] |
|
32 | 32 | }, |
33 | 33 | "source": [ |
34 | 34 | "\n", |
35 | | - "Introducing Unsloth [Standby for RL](https://docs.unsloth.ai/basics/memory-efficient-rl): GRPO is now faster, uses 30% less memory with 2x longer context.\n", |
| 35 | + "Introducing Unsloth [Standby for RL](https://unsloth.ai/docs/basics/memory-efficient-rl): GRPO is now faster, uses 30% less memory with 2x longer context.\n", |
36 | 36 | "\n", |
37 | | - "Gpt-oss fine-tuning now supports 8× longer context with 0 accuracy loss. [Read more](https://docs.unsloth.ai/basics/long-context-gpt-oss-training)\n", |
| 37 | + "Gpt-oss fine-tuning now supports 8× longer context with 0 accuracy loss. [Read more](https://unsloth.ai/docs/basics/long-context-gpt-oss-training)\n", |
38 | 38 | "\n", |
39 | | - "Unsloth now supports Text-to-Speech (TTS) models. Read our [guide here](https://docs.unsloth.ai/basics/text-to-speech-tts-fine-tuning).\n", |
| 39 | + "Unsloth now supports Text-to-Speech (TTS) models. Read our [guide here](https://unsloth.ai/docs/basics/text-to-speech-tts-fine-tuning).\n", |
40 | 40 | "\n", |
41 | | - "Visit our docs for all our [model uploads](https://docs.unsloth.ai/get-started/all-our-models) and [notebooks](https://docs.unsloth.ai/get-started/unsloth-notebooks).\n" |
| 41 | + "Visit our docs for all our [model uploads](https://unsloth.ai/docs/get-started/all-our-models) and [notebooks](https://unsloth.ai/docs/get-started/unsloth-notebooks).\n" |
42 | 42 | ] |
43 | 43 | }, |
44 | 44 | { |
|
9703 | 9703 | "outputs": [], |
9704 | 9704 | "source": [ |
9705 | 9705 | "# Merge to 16bit\n", |
9706 | | - "if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"merged_16bit\",)\n", |
9707 | | - "if False: model.push_to_hub_merged(\"hf/model\", tokenizer, save_method = \"merged_16bit\", token = \"\")\n", |
| 9706 | + "if False: model.save_pretrained_merged(\"advanced_llama_finetune_16bit\", tokenizer, save_method = \"merged_16bit\",)\n", |
| 9707 | + "if False: model.push_to_hub_merged(\"HF_USERNAME/advanced_llama_finetune_16bit\", tokenizer, save_method = \"merged_16bit\", token = \"YOUR_HF_TOKEN\")\n", |
9708 | 9708 | "\n", |
9709 | 9709 | "# Merge to 4bit\n", |
9710 | | - "if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"merged_4bit\",)\n", |
9711 | | - "if False: model.push_to_hub_merged(\"hf/model\", tokenizer, save_method = \"merged_4bit\", token = \"\")\n", |
| 9710 | + "if False: model.save_pretrained_merged(\"advanced_llama_finetune_4bit\", tokenizer, save_method = \"merged_4bit\",)\n", |
| 9711 | + "if False: model.push_to_hub_merged(\"HF_USERNAME/advanced_llama_finetune_4bit\", tokenizer, save_method = \"merged_4bit\", token = \"YOUR_HF_TOKEN\")\n", |
9712 | 9712 | "\n", |
9713 | 9713 | "# Just LoRA adapters\n", |
9714 | 9714 | "if False:\n", |
9715 | | - " model.save_pretrained(\"model\")\n", |
9716 | | - " tokenizer.save_pretrained(\"model\")\n", |
| 9715 | + " model.save_pretrained(\"advanced_llama_lora\")\n", |
| 9716 | + " tokenizer.save_pretrained(\"advanced_llama_lora\")\n", |
9717 | 9717 | "if False:\n", |
9718 | | - " model.push_to_hub(\"hf/model\", token = \"\")\n", |
9719 | | - " tokenizer.push_to_hub(\"hf/model\", token = \"\")\n" |
| 9718 | + " model.push_to_hub(\"HF_USERNAME/advanced_llama_lora\", token = \"YOUR_HF_TOKEN\")\n", |
| 9719 | + " tokenizer.push_to_hub(\"HF_USERNAME/advanced_llama_lora\", token = \"YOUR_HF_TOKEN\")\n" |
9720 | 9720 | ] |
9721 | 9721 | }, |
9722 | 9722 | { |
|
9728 | 9728 | "### GGUF / llama.cpp Conversion\n", |
9729 | 9729 | "To save to `GGUF` / `llama.cpp`, we support it natively now! We clone `llama.cpp` and we default save it to `q8_0`. We allow all methods like `q4_k_m`. Use `save_pretrained_gguf` for local saving and `push_to_hub_gguf` for uploading to HF.\n", |
9730 | 9730 | "\n", |
9731 | | - "Some supported quant methods (full list on our [Wiki page](https://github.com/unslothai/unsloth/wiki#gguf-quantization-options)):\n", |
| 9731 | + "Some supported quant methods (full list on our [docs page](https://github.com/unslothai/unsloth/wiki#gguf-quantization-options)):\n", |
9732 | 9732 | "* `q8_0` - Fast conversion. High resource use, but generally acceptable.\n", |
9733 | 9733 | "* `q4_k_m` - Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K.\n", |
9734 | 9734 | "* `q5_k_m` - Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K.\n", |
|
9745 | 9745 | "outputs": [], |
9746 | 9746 | "source": [ |
9747 | 9747 | "# Save to 8bit Q8_0\n", |
9748 | | - "if False: model.save_pretrained_gguf(\"model\", tokenizer,)\n", |
| 9748 | + "if False: model.save_pretrained_gguf(\"advanced_llama_finetune\", tokenizer,)\n", |
9749 | 9749 | "# Remember to go to https://huggingface.co/settings/tokens for a token!\n", |
9750 | 9750 | "# And change hf to your username!\n", |
9751 | | - "if False: model.push_to_hub_gguf(\"hf/model\", tokenizer, token = \"\")\n", |
| 9751 | + "if False: model.push_to_hub_gguf(\"HF_USERNAME/advanced_llama_finetune\", tokenizer, token = \"YOUR_HF_TOKEN\")\n", |
9752 | 9752 | "\n", |
9753 | 9753 | "# Save to 16bit GGUF\n", |
9754 | | - "if False: model.save_pretrained_gguf(\"model\", tokenizer, quantization_method = \"f16\")\n", |
9755 | | - "if False: model.push_to_hub_gguf(\"hf/model\", tokenizer, quantization_method = \"f16\", token = \"\")\n", |
| 9754 | + "if False: model.save_pretrained_gguf(\"advanced_llama_finetune\", tokenizer, quantization_method = \"f16\")\n", |
| 9755 | + "if False: model.push_to_hub_gguf(\"HF_USERNAME/advanced_llama_finetune\", tokenizer, quantization_method = \"f16\", token = \"YOUR_HF_TOKEN\")\n", |
9756 | 9756 | "\n", |
9757 | 9757 | "# Save to q4_k_m GGUF\n", |
9758 | | - "if False: model.save_pretrained_gguf(\"model\", tokenizer, quantization_method = \"q4_k_m\")\n", |
9759 | | - "if False: model.push_to_hub_gguf(\"hf/model\", tokenizer, quantization_method = \"q4_k_m\", token = \"\")\n", |
| 9758 | + "if False: model.save_pretrained_gguf(\"advanced_llama_finetune\", tokenizer, quantization_method = \"q4_k_m\")\n", |
| 9759 | + "if False: model.push_to_hub_gguf(\"HF_USERNAME/advanced_llama_finetune\", tokenizer, quantization_method = \"q4_k_m\", token = \"YOUR_HF_TOKEN\")\n", |
9760 | 9760 | "\n", |
9761 | 9761 | "# Save to multiple GGUF options - much faster if you want multiple!\n", |
9762 | 9762 | "if False:\n", |
9763 | 9763 | " model.push_to_hub_gguf(\n", |
9764 | | - " \"hf/model\", # Change hf to your username!\n", |
| 9764 | + " \"HF_USERNAME/advanced_llama_finetune\", # Change hf to your username!\n", |
9765 | 9765 | " tokenizer,\n", |
9766 | 9766 | " quantization_method = [\"q4_k_m\", \"q8_0\", \"q5_k_m\",],\n", |
9767 | | - " token = \"\",\n", |
| 9767 | + " token = \"YOUR_HF_TOKEN\",\n", |
9768 | 9768 | " )" |
9769 | 9769 | ] |
9770 | 9770 | }, |
|
9774 | 9774 | "id": "V15Yhj1V9lwG" |
9775 | 9775 | }, |
9776 | 9776 | "source": [ |
9777 | | - "Now, use the `model-unsloth.gguf` file or `model-unsloth-Q4_K_M.gguf` file in llama.cpp.\n", |
| 9777 | + "Now, use the `advanced_llama_finetune.Q8_0.gguf` file or `advanced_llama_finetune.Q4_K_M.gguf` file in llama.cpp.\n", |
9778 | 9778 | "\n", |
9779 | 9779 | "And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/unsloth) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n", |
9780 | 9780 | "\n", |
9781 | 9781 | "Some other links:\n", |
9782 | 9782 | "1. Train your own reasoning model - Llama GRPO notebook [Free Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-GRPO.ipynb)\n", |
9783 | 9783 | "2. Saving finetunes to Ollama. [Free notebook](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3_(8B)-Ollama.ipynb)\n", |
9784 | 9784 | "3. Llama 3.2 Vision finetuning - Radiography use case. [Free Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.2_(11B)-Vision.ipynb)\n", |
9785 | | - "6. See notebooks for DPO, ORPO, Continued pretraining, conversational finetuning and more on our [documentation](https://docs.unsloth.ai/get-started/unsloth-notebooks)!\n", |
| 9785 | + "6. See notebooks for DPO, ORPO, Continued pretraining, conversational finetuning and more on our [documentation](https://unsloth.ai/docs/get-started/unsloth-notebooks)!\n", |
9786 | 9786 | "\n", |
9787 | 9787 | "<div class=\"align-center\">\n", |
9788 | 9788 | " <a href=\"https://unsloth.ai\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n", |
9789 | 9789 | " <a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord.png\" width=\"145\"></a>\n", |
9790 | | - " <a href=\"https://docs.unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a>\n", |
| 9790 | + " <a href=\"https://unsloth.ai/docs/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a>\n", |
9791 | 9791 | "\n", |
9792 | 9792 | " Join Discord if you need help + ⭐️ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐️\n", |
9793 | 9793 | "\n", |
|
0 commit comments