Estrategia eficiente para la mejora de las capacidades de modelos grandes de lenguaje (LLMs)
dc.contributor.advisor | Niño Vásquez, Luis Fernando | spa |
dc.contributor.author | Velandia Gutiérrez, Julián Camilo | spa |
dc.contributor.cvlac | Velandia Gutiérrez, Julián Camilo [0002030716] | spa |
dc.contributor.orcid | Velandia Gutiérrez, Julián Camilo [0009-0000-8617-7445] | spa |
dc.contributor.researchgroup | laboratorio de Investigación en Sistemas Inteligentes Lisi | spa |
dc.date.accessioned | 2025-06-25T15:10:11Z | |
dc.date.available | 2025-06-25T15:10:11Z | |
dc.date.issued | 2025 | |
dc.description | ilustraciones, diagramas, tablas | spa |
dc.description.abstract | Los grandes modelos de lenguaje (LLMs) se han consolidado como un hito en el ámbito de la inteligencia artificial y el procesamiento del lenguaje natural, pero su implementación a gran escala se ve limitada por la necesidad de recursos computacionales elevados. Este trabajo propone que a partir de un modelo base, se exploren y combinen técnicas de procesamiento y selección cuidadosa de datos, entrenamiento y ajustes en la arquitectura, con el fin de mejorar la eficiencia de los modelos en entornos con recursos restringidos y sobre una base de conocimiento delimitada. El enfoque metodológico incluyó la definición de criterios para la elaboración de conjuntos de datos confiables, la experimentación controlada con diferentes configuraciones y la evaluación sistemática de las variantes resultantes en términos de capacidad, versatilidad, tiempo de respuesta y seguridad. Finalmente, se llevaron a cabo pruebas comparativas, midiendo el desempeño de las variantes desarrolladas y validando la eficacia de las estrategias propuestas (Texto tomado de la fuente). | spa |
dc.description.abstract | Large language models (LLMs) have emerged as a milestone in the field of artificial intelligence and natural language processing. However, their large-scale deployment remains constrained by the high computational resources they require. This work proposes that, starting from a base model, a combination of techniques—including careful data processing and selection, training strategies, and architectural adjustments—can be explored to improve model efficiency in resource-constrained environments and within a defined knowledge scope. The methodological approach involved defining criteria for building reliable datasets, conducting controlled experiments with various configurations, and systematically evaluating the resulting model variants in terms of capacity, versatility, response time, and safety. Finally, comparative tests were carried out to measure the performance of the developed variants and validate the effectiveness of the proposed strategies. | eng |
dc.description.degreelevel | Maestría | spa |
dc.description.degreename | Magíster en Ingeniería - Ingeniería de Sistemas y Computación | spa |
dc.description.methods | La metodología propuesta para abordar el problema y alcanzar los objetivos delineados se centra en un enfoque cuantitativo y experimental, mediante el cual se investigarán, compararán y filtrarán métodos de optimización aplicables a grandes modelos de lenguaje. Inicialmente, se llevará a cabo una revisión exhaustiva de la literatura para identificar los métodos de optimización existentes y relevantes. Seguidamente, se establecerán criterios claros para seleccionar aquellos métodos que serán sometidos a prueba, basándose en su relevancia teórica y viabilidad práctica. En la investigación se determinarán los requerimientos específicos de datos para cada método de optimización, abarcando aspectos como el formato, extensión y temáticas de los datos necesarios. En cuanto a los materiales y datos, se utilizarán 1920 tesis del repositorio de la Universidad Nacional de Colombia (UNAL), las cuales serán sometidas a procesos de obtención, limpieza y preparación para asegurar su idoneidad para el entrenamiento de modelos. Este conjunto de datos representa una fuente rica y diversa en contenido, permitiendo evaluar la versatilidad y adaptabilidad de los métodos de optimización en contextos variados. El proceso de limpieza y preparación de datos se diseñará para maximizar la calidad y coherencia de la información, facilitando así la comparación justa entre diferentes técnicas de optimización.La fase experimental consistirá en entrenar un modelo base con distintas combinaciones de métodos de optimización seleccionados. Cada modelo resultante será evaluado a través de pruebas con conjuntos de referencia (benchmarks), versatilidad, eficacia en escenarios de few-shot, peso, tiempo de respuesta y seguridad. Esta evaluación comparativa permitirá determinar las combinaciones de técnicas que ofrecen los mejores equilibrios entre estos rubros, orientando hacia soluciones que mejoren la accesibilidad y eficiencia de los LLMs. Los resultados y conclusiones de esta investigación proporcionarán metodologías valiosas sobre cómo mejorar el rendimiento los LLMs de manera eficiente. | spa |
dc.description.researcharea | Sistemas inteligentes | spa |
dc.format.extent | 65 páginas | spa |
dc.format.mimetype | application/pdf | spa |
dc.identifier.instname | Universidad Nacional de Colombia | spa |
dc.identifier.reponame | Repositorio Institucional Universidad Nacional de Colombia | spa |
dc.identifier.repourl | https://repositorio.unal.edu.co/ | spa |
dc.identifier.uri | https://repositorio.unal.edu.co/handle/unal/88248 | |
dc.language.iso | spa | spa |
dc.publisher | Universidad Nacional de Colombia | spa |
dc.publisher.branch | Universidad Nacional de Colombia - Sede Bogotá | spa |
dc.publisher.faculty | Facultad de Ingeniería | spa |
dc.publisher.place | Bogotá, Colombia | spa |
dc.publisher.program | Bogotá - Ingeniería - Maestría en Ingeniería - Ingeniería de Sistemas y Computación | spa |
dc.relation.references | Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv:2303.18223, 2023. | spa |
dc.relation.references | Sourab Mangrulkar and Sylvain Gugger and Lysandre Debut and Younes Belkada and Sayak Paul and Benjamin Bossan, State-of-the-art Parameter-Efficient Fine-Tuning (PEFT) methods, 2022. | spa |
dc.relation.references | Andrej Karpathy, Conference: State of GPT | BRK216HFS, Microsoft Developer, recuperado de: https://www.youtube.com/watch?v=bZQun8Y4L2A, 2023. | spa |
dc.relation.references | Xuan-Phi Nguyen, Sharifah Mahani Aljunied, Shafiq Joty, Lidong Bing, Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically Diverse Prompts, arXiv:2306.11372, 2023. | spa |
dc.relation.references | Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, LoRA: Low-Rank Adaptation of Large Language Models, arXiv:2106.09685, 2021. | spa |
dc.relation.references | Direct Preference Optimization: Your Language Model is Secretly a Reward Model, arXiv:2305.18290, 2023. | spa |
dc.relation.references | Ziegler, Daniel M.; Stiennon, Nisan; Wu, Jeffrey; Brown, Tom B.; Radford, Alec; Amodei, Dario; Christiano, Paul; Irving, Geoffrey. "Fine-Tuning Language Models from Human Preferences". arXiv:1909.08593, 2019 | spa |
dc.relation.references | Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv:2005.11401, 2023 | spa |
dc.relation.references | Haoyu Han, Yu Wang, Harry Shomer, Kai Guo, Jiayuan Ding, Yongjia Lei, Mahantesh Halappanavar, Ryan A. Rossi, Subhabrata Mukherjee, Xianfeng Tang, Qi He, Zhigang Hua, Bo Long, Tong Zhao, Neil Shah, Amin Javari, Yinglong Xia, Jiliang Tang, Retrieval-Augmented Generation with Graphs (GraphRAG), arXiv:2501.00309 | spa |
dc.relation.references | Youyang Ng, Daisuke Miyashita, Yasuto Hoshi, Yasuhiro Morioka, Osamu Torii, Tomoya Kodama, Jun Deguchi, SimplyRetrieve: A Private and Lightweight Retrieval-Centric Generative AI Tool, arXiv:2308.03983, 2023. | spa |
dc.relation.references | Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, Xiaojuan Qi, BiLLM: Pushing the Limit of Post-Training Quantization for LLMs, arXiv:2402.04291 | spa |
dc.relation.references | Xuan-Phi Nguyen, Sharifah Mahani Aljunied, Shafiq Joty, Lidong Bing, Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically Diverse Prompts, arXiv:2306.11372, 2023. | spa |
dc.relation.references | Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, Attention Is All You Need, arXiv:1706.03762, 2017 | spa |
dc.relation.references | Oluwasegun Adedugbe, Elhadj Benkhelifa, Anoud Bani-Hani, A Cloud Computing Capability Model for LargeScale Semantic Annotation, arXiv:2006.13893, 2020 | spa |
dc.relation.references | Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, arXiv:1810.04805, 2019. | spa |
dc.relation.references | Bowen Tan, Zichao Yang, Maruan AI-Shedivat, Eric P. Xing, Zhiting Hu, Progressive Generation of Long Text with Pretrained Language Models, arXiv:2006.15720, 2021. | spa |
dc.relation.references | Egor Lakomkin, Chunyang Wu, Yassir Fathullah, Ozlem Kalinli, Michael L. Seltzer, Christian Fuegen, End-toEnd Speech Recognition Contextualization with Large Language Models, arXiv:2309.10917, 2023. | spa |
dc.relation.references | Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, Robert McHardy, Challenges and Applications of Large Language Models, arXiv:2307.10169, 2023 | spa |
dc.relation.references | frequently asked questions, Amazon EC2 documentation, recuperado de: https://aws.amazon.com/es/ec2/faqs/, 2023 | spa |
dc.relation.references | Brown, Tom B.; Mann, Benjamin; Ryder, Nick; Subbiah, Melanie; Kaplan, Jared; Dhariwal, Prafulla; Neelakantan, Arvind; Shyam, Pranav; Sastry, Girish; Askell, Amanda; Agarwal, Sandhini; Herbert-Voss, Ariel; Krueger, Gretchen; Henighan, Tom; Child, Rewon; Ramesh, Aditya; Ziegler, Daniel M.; Wu, Jeffrey; Winter, Clemens; Hesse, Christopher; Chen, Mark; Sigler, Eric; Litwin, Mateusz; Gray, Scott; Chess, Benjamin; Clark, Jack; Berner, Christopher; McCandlish, Sam; Radford, Alec; Sutskever, Ilya; Amodei. "Language Models are Few-Shot Learners". arXiv:2005.14165, 2021 | spa |
dc.relation.references | Daniel Adiwardana, Thang Luong, Google Research Blog, Towards a Conversational Agent that Can Chat About...Anything, recuperado de: https://blog.research.google/2020/01/towards-conversational-agent-thatcan.html, 2023 | spa |
dc.relation.references | Google AI, AI ACROSS GOOGLE: PaLM 2, recuperado de: https://ai.google/discover/palm2/, 2023 | spa |
dc.relation.references | Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Siddhant Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, Laurent Sifre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Mensch, Jean-Baptiste Lespiau, Maria Tsimpoukelli, Nikolai Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d'Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy, Chris Jones, James Bradbury, Matthew Johnson, Blake Hechtman, Laura Weidinger, Iason Gabriel, William Isaac, Ed Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hassabis, Koray Kavukcuoglu, Geoffrey Irving, Scaling Language Models: Methods, Analysis & Insights from Training Gopher, arXiv:2112.11446, 2021 | spa |
dc.relation.references | Jack Rae, Geoffrey Irving, Laura Weidinger, Google DeepMind: Language modelling at scale: Gopher, ethical considerations, and retrieval, recuperado de: https://deepmind.google/discover/blog/language-modelling-atscale-gopher-ethical-consider ations-and-retrieval/, 2023 | spa |
dc.relation.references | Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, Xia Hu, Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond, arXiv:2304.13712, 2023. | spa |
dc.relation.references | Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer, QLoRA: Efficient Finetuning of Quantized LLMs, arXiv:2305.14314, 2023. | spa |
dc.relation.references | Yixin Liu, Avi Singh, C. Daniel Freeman, John D. Co-Reyes, Peter J. Liu, Improving Large Language Model Fine-Tuning for Solving Math Problems, arXiv:2310.10047, 2023 | spa |
dc.relation.references | David McCandless, Tom Evans, Paul Barton, The Rise and Rise of A.I. Large Language Models (LLMs)& their associated bots like ChatGPT, Recuperado de: https://informationisbeautiful.net/visualizations/the-riseof-generative-ai-large-language-mo dels-llms-like-chatgpt/, 2023 | spa |
dc.relation.references | Cleeremans, A., Servan-Schreiber, D., and McClelland, J. L. Finite-state automata and simple recurrent networks. Neural Computation, 1:372-381, 1989 | spa |
dc.relation.references | Philipp Schmid, Web Tutorial: Train LLMs using QLoRA on Amazon SageMaker, Recuperado de: https://www.philschmid.de/sagemaker-falcon-qlora, 2023 | spa |
dc.relation.references | Zekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhu Chen, Jie Fu, Junran Peng, RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models, arXiv:2310.00746, 2023 | spa |
dc.relation.references | Seonghyeon Ye, Hyeonbin Hwang, Sohee Yang, Hyeongu Yun, Yireun Kim, Minjoon Seo, In-Context Instruction Learning, arXiv:2302.14691, 2023 | spa |
dc.relation.references | Zekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhu Chen, Jie Fu, Junran Peng, RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models, arXiv:2310.00746, 2023 | spa |
dc.relation.references | Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, Zhendong Mao, ExpertPrompting: Instructing Large Language Models to be Distinguished Experts, arXiv:2305.14688, 2023 | spa |
dc.relation.references | Benfeng Xu, An Yang, ExpertLLaMA: Answering Instructions Like an Expert, Recuperado de: https://github.com/OFA-Sys/ExpertLLaMA, 2023 | spa |
dc.relation.references | Simeng Sun, Yang Liu, Dan Iter, Chenguang Zhu, Mohit Iyyer, How Does In-Context Learning Help Prompt Tuning?, arXiv:2302.11521, 2023 | spa |
dc.relation.references | Rishabh Bhardwaj, Soujanya Poria, Language Model Unalignment: Parametric Red-Teaming to Expose Hidden Harms and Biases, arXiv:2310.14303, 2023 | spa |
dc.relation.references | Pascanu, Razvan; Mikolov, Tomas; Bengio, Yoshua, "On the difficulty of training Recurrent Neural Networks". arXiv:1211.5063, 2012 | spa |
dc.relation.references | Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, Yang Zhang, "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models, arXiv:2308.03825, 2023 | spa |
dc.relation.references | Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimism, Zihao Li, Zhuoran Yang, Mengdi Wang, arXiv:2305.18438, 2023 | spa |
dc.relation.references | mrbullwinkle, eric-urban, Microsoft Azure Blog, Planeamiento de red teaming para modelos de lenguaje grandes (LLM) y sus aplicaciones, recuperado de: https://learn.microsoft.com/es-es/azure/aiservices/openai/concepts/red-teaming, 2023 | spa |
dc.relation.references | Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda, Holistic Evaluation of Language Models, arXiv:2211.09110, 2022 | spa |
dc.relation.references | Lianmin Zheng and Wei-Lin Chiang and Ying Sheng and Siyuan Zhuang and Zhanghao Wu and Yonghao Zhuang and Zi Lin and Zhuohan Li and Dacheng Li and Eric. P Xing and Hao Zhang and Joseph E. Gonzalez and Ion Stoica, Judging LLM-as-a-judge with MT-Bench and Chatbot Arena, arXiv: 2306.05685, 2023 | spa |
dc.relation.references | Lianmin Zheng, Ying Sheng, Wei-Lin Chiang, Hao Zhang, Joseph E. Gonzalez, Ion Stoica, Chatbot Arena: Benchmarking LLMs in the Wild with Elo Ratings, Recuperado de: https://lmsys.org/blog/2023-05-03-arena/ | spa |
dc.relation.references | Giménez Fayos, María Teresa, Una aproximación basada en aprendizaje automático para diversos problemas de procesamiento de lenguaje natural en redes sociales, 2016 | spa |
dc.relation.references | Xi Tian, Web Tutorial: Fine-Tuning Falcon LLM 7B/40B, recuperado de: https://lambdalabs.com/blog/finetuning-falcon-llm-7b/40b, 2023 | spa |
dc.relation.references | 1littlecoder, Web Tutorial: Falcon-7B-Instruct LLM with LangChain Tutorial, Recuperado de: https://www.youtube.com/watch?v=mAoNANPOsd0, 2023 | spa |
dc.relation.references | AWS, Official Documentation: Instancias P3 de Amazon EC2, Recuperado de: https://aws.amazon.com/es/ec2/instance-types/p3/ | spa |
dc.relation.references | Abi Aryan, Aakash Kumar Nain, Andrew McMahon, Lucas Augusto Meyer, Harpreet Singh Sahota, The Costly Dilemma: Generalization, Evaluation and Cost-Optimal Deployment of Large Language Models, arXiv:2308.08061, 2023 | spa |
dc.relation.references | Ariel N. Lee, Cole J. Hunter, Nataniel Ruiz, Platypus: Quick, Cheap, and Powerful Refinement of LLMs, arXiv:2308.07317, 2023. | spa |
dc.relation.references | E. Almazrouei, H. Alobeidli, A. Alshamsi, A. Cappelli, R. Cojocaru, M. Debbah, E. Goffinet, D. Heslow, J. Launay, Q. Malartic, B. Noune, B. Pannier, and G. Penedo. Falcon-40b: an open large language model with state-of-the-art performance, 2023. | spa |
dc.relation.references | H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M.-A. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023. | spa |
dc.relation.references | Institutional Repository of Universidad Nacional / Trabajos de Grado, recuperado de: https://repositorio.unal.edu.co/handle/unal/5 | spa |
dc.relation.references | Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu Richard Socher, Xavier Amatriain, Jianfeng Gao, Large Language Models: A Survey, arXiv: 2402.06196 | spa |
dc.relation.references | Luis Fernando Niño Vasquez, Laboratorio de investigación en sistemas inteligentes - lisi, Recuperado de: http://www.hermes.unal.edu.co/pages/Consultas/Grupo.xhtml?idGrupo=409&opcion=1 | spa |
dc.relation.references | Rohan Taori and Ishaan Gulrajani and Tianyi Zhang and Yann Dubois and Xuechen Li and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto, Alpaca: A Strong, Replicable Instruction-Following Model, Recuperado de: https://crfm.stanford.edu/2023/03/13/alpaca.html, 2022 | spa |
dc.relation.references | Julien Launay, TII Falcon LLM License Version 1.0, Recuperado de: https://github.com/DecentralisedAI/falcon-40b/blob/main/LICENSE.txt, 2023 | spa |
dc.relation.references | MetaAI, Llama License, Recuperado de: https://ai.meta.com/llama/license/, 2023 | spa |
dc.relation.references | Cobus Greyling, Fine-Tuning LLMs With Retrieval Augmented Generation (RAG), Recuperado de: https://cobusgreyling.medium.com/fine-tuning-llms-with-retrieval-augmented-generation-r ag-c66e56aec858 | spa |
dc.relation.references | MatrixFlows, RAG, Fine-Tuning or Both? A Complete Framework for Choosing the Right Strategy, Recuperado de: https://www.matrixflows.com/blog/retrieval-augmented-generation-rag-finetuning-hybrid-f ramework-for-choosing-right-strategy | spa |
dc.relation.references | Harsha Srivatsa, Fine-Tuning versus RAG in Generative AI Applications Architecture, Recuperado de: https://harsha-srivatsa.medium.com/fine-tuning-versus-rag-in-generative-ai-applications-ar chitecture-d54ca6d2acb8 | spa |
dc.relation.references | Zifei Xu, Alexander Lan, Wanzin Yazar, Tristan Webb, Sayeh Sharify, Xin Wang, Scaling Laws for Post Training Quantized Large Language Models, arXiv:2410.12119 | spa |
dc.relation.references | Graphical Markov models: overview, Nanny Wermuth, D.R. Cox, arXiv:1407.7783, 2014 | spa |
dc.relation.references | Hochreiter Sepp, Schmidhuber Jürgen, Long Short-term Memory, 10.1162/neco.1997.9.8.1735, 1997 | spa |
dc.relation.references | Wenhua Cheng, Weiwei Zhang, Haihao Shen, Yiyang Cai, Xin He, Kaokao Lv, Yi Liu, Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs, arXiv:2309.05516 | spa |
dc.relation.references | Zhihang Yuan, Jiawei Liu, Jiaxiang Wu, Dawei Yang, Qiang Wu, Guangyu Sun, Wenyu Liu, Xinggang Wang, Bingzhe Wu, Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performance, arXiv:2303.13003 | spa |
dc.relation.references | Shengwei Xu, Yuxuan Lu, Grant Schoenebeck, Yuqing Kong, Benchmarking LLMs' Judgments with No Gold Standard, arXiv:2411.07127 | spa |
dc.relation.references | Hongyin Luo & Wei Sun, Addition is All You Need for Energy-Efficient Language Models, arXiv:2410.00907v2 | spa |
dc.relation.references | Aditi Singh, Abul Ehtesham, Saket Kumar, Tala Talaei Khoei, Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG, arXiv:2501.09136 | spa |
dc.relation.references | Marc Pickett, Jeremy Hartman, Ayan Kumar Bhowmick, Raquib-ul Alam, Aditya Vempaty, Better RAG using Relevant Information Gain, arXiv:2407.12101 | spa |
dc.relation.references | Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed, Mistral 7B, arXiv:2310.06825 | spa |
dc.relation.references | Apache Software foundation, Apache License, Version 2.0, https://www.apache.org/licenses/LICENSE-2.0 | spa |
dc.relation.references | Open Source Initiative, https://opensource.org/license/mit | spa |
dc.relation.references | DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z.F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J.L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R.J. Chen, R.L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S.S. Li et al. (100 additional authors not shown), DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, arXiv:2501.12948 | spa |
dc.relation.references | Aitor Arrieta, Miriam Ugarte, Pablo Valle, José Antonio Parejo, Sergio Segura, o3-mini vs DeepSeek-R1: Which One is Safer?, arXiv:2501.18438 | spa |
dc.relation.references | An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zihan Qiu, Qwen2.5 Technical Report, arXiv:2412.15115 | spa |
dc.relation.references | Isha Chaudhary, Qian Hu, Manoj Kumar, Morteza Ziyadi, Rahul Gupta, Gagandeep Singh, Quantitative Certification of Bias in Large Language Models, arXiv:2405.18780 | spa |
dc.relation.references | Multi-task Language Understanding on MMLU, https://paperswithcode.com/sota/multi-task-language-understanding-on-mmlu | spa |
dc.relation.references | Wolfram Ravenwolf, LLM Comparison/Test: 25 SOTA LLMs (including QwQ) through 59 MMLU-Pro CS benchmark runs, https://huggingface.co/blog/wolfram/llm-comparison-test-2024-12-04 | spa |
dc.relation.references | Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, Jian Guo, A Survey on LLM-as-a-Judge, arXiv:2411.15594 | spa |
dc.relation.references | Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, Thomas Back, Reasoning with Large Language Models, a Survey, arXiv:2407.11511 | spa |
dc.rights.accessrights | info:eu-repo/semantics/openAccess | spa |
dc.rights.license | Reconocimiento 4.0 Internacional | spa |
dc.rights.uri | http://creativecommons.org/licenses/by/4.0/ | spa |
dc.subject.ddc | 000 - Ciencias de la computación, información y obras generales::004 - Procesamiento de datos Ciencia de los computadores | spa |
dc.subject.ddc | 000 - Ciencias de la computación, información y obras generales::001 - Conocimiento | spa |
dc.subject.ddc | 000 - Ciencias de la computación, información y obras generales::003 - Sistemas | spa |
dc.subject.lemb | LENGUAJES NATURALES | spa |
dc.subject.lemb | Natural languages | eng |
dc.subject.lemb | LENGUAJES DE MAQUINA | spa |
dc.subject.lemb | Programming languages | eng |
dc.subject.lemb | LENGUAJES DE PROGRAMACION (COMPUTADORES ELECTRONICOS) | spa |
dc.subject.lemb | Programming languages (electronic computers) | eng |
dc.subject.lemb | PROCESAMIENTO ELECTRONICO DE DATOS | spa |
dc.subject.lemb | Electronic data processing | eng |
dc.subject.lemb | LINGUISTICA COMPUTACIONAL | spa |
dc.subject.lemb | Computational linguistics | eng |
dc.subject.lemb | LEXICOGRAFIA-PROCESAMIENTO DE DATOS | spa |
dc.subject.lemb | Lexicography Data processing | eng |
dc.subject.lemb | APRENDIZAJE AUTOMATICO (INTELIGENCIA ARTIFICIAL) | spa |
dc.subject.lemb | Machine learning | eng |
dc.subject.lemb | INTELIGENCIA ARTIFICIAL-PROCESAMIENTO DE DATOS | spa |
dc.subject.lemb | Artificial intelligen - data processing | eng |
dc.subject.proposal | Grandes Modelos de Lenguaje (LLMs) | spa |
dc.subject.proposal | Eficiencia computacional | spa |
dc.subject.proposal | Entrenamiento eficiente | spa |
dc.subject.proposal | Benchmarks de Modelos de Lenguaje | spa |
dc.subject.proposal | Large Language Models (LLMs) | eng |
dc.subject.proposal | Computational Efficiency | eng |
dc.subject.proposal | Efficient Training | eng |
dc.subject.proposal | Language Model Benchmarks | eng |
dc.subject.wikidata | Semantic Web | eng |
dc.subject.wikidata | Web semántica | spa |
dc.title | Estrategia eficiente para la mejora de las capacidades de modelos grandes de lenguaje (LLMs) | spa |
dc.title.translated | Efficient strategy for improving the capabilities of large language models (LLMs) | eng |
dc.type | Trabajo de grado - Maestría | spa |
dc.type.coar | http://purl.org/coar/resource_type/c_bdcc | spa |
dc.type.coarversion | http://purl.org/coar/version/c_ab4af688f83e57aa | spa |
dc.type.content | Text | spa |
dc.type.driver | info:eu-repo/semantics/masterThesis | spa |
dc.type.redcol | http://purl.org/redcol/resource_type/TM | spa |
dc.type.version | info:eu-repo/semantics/acceptedVersion | spa |
dcterms.audience.professionaldevelopment | Estudiantes | spa |
oaire.accessrights | http://purl.org/coar/access_right/c_abf2 | spa |
oaire.fundername | Julián Camilo Velandia Gutiérrez | spa |
Archivos
Bloque original
1 - 1 de 1
Cargando...
- Nombre:
- DocumentoFinal1026598964 (1).pdf
- Tamaño:
- 3.84 MB
- Formato:
- Adobe Portable Document Format
- Descripción:
- Tesis de Maestría en Ingeniería de Sistemas y Computación
Bloque de licencias
1 - 1 de 1
Cargando...
- Nombre:
- license.txt
- Tamaño:
- 5.74 KB
- Formato:
- Item-specific license agreed upon to submission
- Descripción: