Estrategia eficiente para la mejora de las capacidades de modelos grandes de lenguaje (LLMs)

dc.contributor.advisorNiño Vásquez, Luis Fernandospa
dc.contributor.authorVelandia Gutiérrez, Julián Camilospa
dc.contributor.cvlacVelandia Gutiérrez, Julián Camilo [0002030716]spa
dc.contributor.orcidVelandia Gutiérrez, Julián Camilo [0009-0000-8617-7445]spa
dc.contributor.researchgrouplaboratorio de Investigación en Sistemas Inteligentes Lisispa
dc.date.accessioned2025-06-25T15:10:11Z
dc.date.available2025-06-25T15:10:11Z
dc.date.issued2025
dc.descriptionilustraciones, diagramas, tablasspa
dc.description.abstractLos grandes modelos de lenguaje (LLMs) se han consolidado como un hito en el ámbito de la inteligencia artificial y el procesamiento del lenguaje natural, pero su implementación a gran escala se ve limitada por la necesidad de recursos computacionales elevados. Este trabajo propone que a partir de un modelo base, se exploren y combinen técnicas de procesamiento y selección cuidadosa de datos, entrenamiento y ajustes en la arquitectura, con el fin de mejorar la eficiencia de los modelos en entornos con recursos restringidos y sobre una base de conocimiento delimitada. El enfoque metodológico incluyó la definición de criterios para la elaboración de conjuntos de datos confiables, la experimentación controlada con diferentes configuraciones y la evaluación sistemática de las variantes resultantes en términos de capacidad, versatilidad, tiempo de respuesta y seguridad. Finalmente, se llevaron a cabo pruebas comparativas, midiendo el desempeño de las variantes desarrolladas y validando la eficacia de las estrategias propuestas (Texto tomado de la fuente).spa
dc.description.abstractLarge language models (LLMs) have emerged as a milestone in the field of artificial intelligence and natural language processing. However, their large-scale deployment remains constrained by the high computational resources they require. This work proposes that, starting from a base model, a combination of techniques—including careful data processing and selection, training strategies, and architectural adjustments—can be explored to improve model efficiency in resource-constrained environments and within a defined knowledge scope. The methodological approach involved defining criteria for building reliable datasets, conducting controlled experiments with various configurations, and systematically evaluating the resulting model variants in terms of capacity, versatility, response time, and safety. Finally, comparative tests were carried out to measure the performance of the developed variants and validate the effectiveness of the proposed strategies.eng
dc.description.degreelevelMaestríaspa
dc.description.degreenameMagíster en Ingeniería - Ingeniería de Sistemas y Computaciónspa
dc.description.methodsLa metodología propuesta para abordar el problema y alcanzar los objetivos delineados se centra en un enfoque cuantitativo y experimental, mediante el cual se investigarán, compararán y filtrarán métodos de optimización aplicables a grandes modelos de lenguaje. Inicialmente, se llevará a cabo una revisión exhaustiva de la literatura para identificar los métodos de optimización existentes y relevantes. Seguidamente, se establecerán criterios claros para seleccionar aquellos métodos que serán sometidos a prueba, basándose en su relevancia teórica y viabilidad práctica. En la investigación se determinarán los requerimientos específicos de datos para cada método de optimización, abarcando aspectos como el formato, extensión y temáticas de los datos necesarios. En cuanto a los materiales y datos, se utilizarán 1920 tesis del repositorio de la Universidad Nacional de Colombia (UNAL), las cuales serán sometidas a procesos de obtención, limpieza y preparación para asegurar su idoneidad para el entrenamiento de modelos. Este conjunto de datos representa una fuente rica y diversa en contenido, permitiendo evaluar la versatilidad y adaptabilidad de los métodos de optimización en contextos variados. El proceso de limpieza y preparación de datos se diseñará para maximizar la calidad y coherencia de la información, facilitando así la comparación justa entre diferentes técnicas de optimización.La fase experimental consistirá en entrenar un modelo base con distintas combinaciones de métodos de optimización seleccionados. Cada modelo resultante será evaluado a través de pruebas con conjuntos de referencia (benchmarks), versatilidad, eficacia en escenarios de few-shot, peso, tiempo de respuesta y seguridad. Esta evaluación comparativa permitirá determinar las combinaciones de técnicas que ofrecen los mejores equilibrios entre estos rubros, orientando hacia soluciones que mejoren la accesibilidad y eficiencia de los LLMs. Los resultados y conclusiones de esta investigación proporcionarán metodologías valiosas sobre cómo mejorar el rendimiento los LLMs de manera eficiente.spa
dc.description.researchareaSistemas inteligentesspa
dc.format.extent65 páginasspa
dc.format.mimetypeapplication/pdfspa
dc.identifier.instnameUniversidad Nacional de Colombiaspa
dc.identifier.reponameRepositorio Institucional Universidad Nacional de Colombiaspa
dc.identifier.repourlhttps://repositorio.unal.edu.co/spa
dc.identifier.urihttps://repositorio.unal.edu.co/handle/unal/88248
dc.language.isospaspa
dc.publisherUniversidad Nacional de Colombiaspa
dc.publisher.branchUniversidad Nacional de Colombia - Sede Bogotáspa
dc.publisher.facultyFacultad de Ingenieríaspa
dc.publisher.placeBogotá, Colombiaspa
dc.publisher.programBogotá - Ingeniería - Maestría en Ingeniería - Ingeniería de Sistemas y Computaciónspa
dc.relation.referencesWayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv:2303.18223, 2023.spa
dc.relation.referencesSourab Mangrulkar and Sylvain Gugger and Lysandre Debut and Younes Belkada and Sayak Paul and Benjamin Bossan, State-of-the-art Parameter-Efficient Fine-Tuning (PEFT) methods, 2022.spa
dc.relation.referencesAndrej Karpathy, Conference: State of GPT | BRK216HFS, Microsoft Developer, recuperado de: https://www.youtube.com/watch?v=bZQun8Y4L2A, 2023.spa
dc.relation.referencesXuan-Phi Nguyen, Sharifah Mahani Aljunied, Shafiq Joty, Lidong Bing, Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically Diverse Prompts, arXiv:2306.11372, 2023.spa
dc.relation.referencesEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, LoRA: Low-Rank Adaptation of Large Language Models, arXiv:2106.09685, 2021.spa
dc.relation.referencesDirect Preference Optimization: Your Language Model is Secretly a Reward Model, arXiv:2305.18290, 2023.spa
dc.relation.referencesZiegler, Daniel M.; Stiennon, Nisan; Wu, Jeffrey; Brown, Tom B.; Radford, Alec; Amodei, Dario; Christiano, Paul; Irving, Geoffrey. "Fine-Tuning Language Models from Human Preferences". arXiv:1909.08593, 2019spa
dc.relation.referencesPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv:2005.11401, 2023spa
dc.relation.referencesHaoyu Han, Yu Wang, Harry Shomer, Kai Guo, Jiayuan Ding, Yongjia Lei, Mahantesh Halappanavar, Ryan A. Rossi, Subhabrata Mukherjee, Xianfeng Tang, Qi He, Zhigang Hua, Bo Long, Tong Zhao, Neil Shah, Amin Javari, Yinglong Xia, Jiliang Tang, Retrieval-Augmented Generation with Graphs (GraphRAG), arXiv:2501.00309spa
dc.relation.referencesYouyang Ng, Daisuke Miyashita, Yasuto Hoshi, Yasuhiro Morioka, Osamu Torii, Tomoya Kodama, Jun Deguchi, SimplyRetrieve: A Private and Lightweight Retrieval-Centric Generative AI Tool, arXiv:2308.03983, 2023.spa
dc.relation.referencesWei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, Xiaojuan Qi, BiLLM: Pushing the Limit of Post-Training Quantization for LLMs, arXiv:2402.04291spa
dc.relation.referencesXuan-Phi Nguyen, Sharifah Mahani Aljunied, Shafiq Joty, Lidong Bing, Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically Diverse Prompts, arXiv:2306.11372, 2023.spa
dc.relation.referencesAshish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, Attention Is All You Need, arXiv:1706.03762, 2017spa
dc.relation.referencesOluwasegun Adedugbe, Elhadj Benkhelifa, Anoud Bani-Hani, A Cloud Computing Capability Model for LargeScale Semantic Annotation, arXiv:2006.13893, 2020spa
dc.relation.referencesJacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, arXiv:1810.04805, 2019.spa
dc.relation.referencesBowen Tan, Zichao Yang, Maruan AI-Shedivat, Eric P. Xing, Zhiting Hu, Progressive Generation of Long Text with Pretrained Language Models, arXiv:2006.15720, 2021.spa
dc.relation.referencesEgor Lakomkin, Chunyang Wu, Yassir Fathullah, Ozlem Kalinli, Michael L. Seltzer, Christian Fuegen, End-toEnd Speech Recognition Contextualization with Large Language Models, arXiv:2309.10917, 2023.spa
dc.relation.referencesJean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, Robert McHardy, Challenges and Applications of Large Language Models, arXiv:2307.10169, 2023spa
dc.relation.referencesfrequently asked questions, Amazon EC2 documentation, recuperado de: https://aws.amazon.com/es/ec2/faqs/, 2023spa
dc.relation.referencesBrown, Tom B.; Mann, Benjamin; Ryder, Nick; Subbiah, Melanie; Kaplan, Jared; Dhariwal, Prafulla; Neelakantan, Arvind; Shyam, Pranav; Sastry, Girish; Askell, Amanda; Agarwal, Sandhini; Herbert-Voss, Ariel; Krueger, Gretchen; Henighan, Tom; Child, Rewon; Ramesh, Aditya; Ziegler, Daniel M.; Wu, Jeffrey; Winter, Clemens; Hesse, Christopher; Chen, Mark; Sigler, Eric; Litwin, Mateusz; Gray, Scott; Chess, Benjamin; Clark, Jack; Berner, Christopher; McCandlish, Sam; Radford, Alec; Sutskever, Ilya; Amodei. "Language Models are Few-Shot Learners". arXiv:2005.14165, 2021spa
dc.relation.referencesDaniel Adiwardana, Thang Luong, Google Research Blog, Towards a Conversational Agent that Can Chat About...Anything, recuperado de: https://blog.research.google/2020/01/towards-conversational-agent-thatcan.html, 2023spa
dc.relation.referencesGoogle AI, AI ACROSS GOOGLE: PaLM 2, recuperado de: https://ai.google/discover/palm2/, 2023spa
dc.relation.referencesJack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Siddhant Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, Laurent Sifre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Mensch, Jean-Baptiste Lespiau, Maria Tsimpoukelli, Nikolai Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d'Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy, Chris Jones, James Bradbury, Matthew Johnson, Blake Hechtman, Laura Weidinger, Iason Gabriel, William Isaac, Ed Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hassabis, Koray Kavukcuoglu, Geoffrey Irving, Scaling Language Models: Methods, Analysis & Insights from Training Gopher, arXiv:2112.11446, 2021spa
dc.relation.referencesJack Rae, Geoffrey Irving, Laura Weidinger, Google DeepMind: Language modelling at scale: Gopher, ethical considerations, and retrieval, recuperado de: https://deepmind.google/discover/blog/language-modelling-atscale-gopher-ethical-consider ations-and-retrieval/, 2023spa
dc.relation.referencesJingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, Xia Hu, Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond, arXiv:2304.13712, 2023.spa
dc.relation.referencesTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer, QLoRA: Efficient Finetuning of Quantized LLMs, arXiv:2305.14314, 2023.spa
dc.relation.referencesYixin Liu, Avi Singh, C. Daniel Freeman, John D. Co-Reyes, Peter J. Liu, Improving Large Language Model Fine-Tuning for Solving Math Problems, arXiv:2310.10047, 2023spa
dc.relation.referencesDavid McCandless, Tom Evans, Paul Barton, The Rise and Rise of A.I. Large Language Models (LLMs)& their associated bots like ChatGPT, Recuperado de: https://informationisbeautiful.net/visualizations/the-riseof-generative-ai-large-language-mo dels-llms-like-chatgpt/, 2023spa
dc.relation.referencesCleeremans, A., Servan-Schreiber, D., and McClelland, J. L. Finite-state automata and simple recurrent networks. Neural Computation, 1:372-381, 1989spa
dc.relation.referencesPhilipp Schmid, Web Tutorial: Train LLMs using QLoRA on Amazon SageMaker, Recuperado de: https://www.philschmid.de/sagemaker-falcon-qlora, 2023spa
dc.relation.referencesZekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhu Chen, Jie Fu, Junran Peng, RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models, arXiv:2310.00746, 2023spa
dc.relation.referencesSeonghyeon Ye, Hyeonbin Hwang, Sohee Yang, Hyeongu Yun, Yireun Kim, Minjoon Seo, In-Context Instruction Learning, arXiv:2302.14691, 2023spa
dc.relation.referencesZekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhu Chen, Jie Fu, Junran Peng, RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models, arXiv:2310.00746, 2023spa
dc.relation.referencesBenfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, Zhendong Mao, ExpertPrompting: Instructing Large Language Models to be Distinguished Experts, arXiv:2305.14688, 2023spa
dc.relation.referencesBenfeng Xu, An Yang, ExpertLLaMA: Answering Instructions Like an Expert, Recuperado de: https://github.com/OFA-Sys/ExpertLLaMA, 2023spa
dc.relation.referencesSimeng Sun, Yang Liu, Dan Iter, Chenguang Zhu, Mohit Iyyer, How Does In-Context Learning Help Prompt Tuning?, arXiv:2302.11521, 2023spa
dc.relation.referencesRishabh Bhardwaj, Soujanya Poria, Language Model Unalignment: Parametric Red-Teaming to Expose Hidden Harms and Biases, arXiv:2310.14303, 2023spa
dc.relation.referencesPascanu, Razvan; Mikolov, Tomas; Bengio, Yoshua, "On the difficulty of training Recurrent Neural Networks". arXiv:1211.5063, 2012spa
dc.relation.referencesXinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, Yang Zhang, "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models, arXiv:2308.03825, 2023spa
dc.relation.referencesReinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimism, Zihao Li, Zhuoran Yang, Mengdi Wang, arXiv:2305.18438, 2023spa
dc.relation.referencesmrbullwinkle, eric-urban, Microsoft Azure Blog, Planeamiento de red teaming para modelos de lenguaje grandes (LLM) y sus aplicaciones, recuperado de: https://learn.microsoft.com/es-es/azure/aiservices/openai/concepts/red-teaming, 2023spa
dc.relation.referencesPercy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda, Holistic Evaluation of Language Models, arXiv:2211.09110, 2022spa
dc.relation.referencesLianmin Zheng and Wei-Lin Chiang and Ying Sheng and Siyuan Zhuang and Zhanghao Wu and Yonghao Zhuang and Zi Lin and Zhuohan Li and Dacheng Li and Eric. P Xing and Hao Zhang and Joseph E. Gonzalez and Ion Stoica, Judging LLM-as-a-judge with MT-Bench and Chatbot Arena, arXiv: 2306.05685, 2023spa
dc.relation.referencesLianmin Zheng, Ying Sheng, Wei-Lin Chiang, Hao Zhang, Joseph E. Gonzalez, Ion Stoica, Chatbot Arena: Benchmarking LLMs in the Wild with Elo Ratings, Recuperado de: https://lmsys.org/blog/2023-05-03-arena/spa
dc.relation.referencesGiménez Fayos, María Teresa, Una aproximación basada en aprendizaje automático para diversos problemas de procesamiento de lenguaje natural en redes sociales, 2016spa
dc.relation.referencesXi Tian, Web Tutorial: Fine-Tuning Falcon LLM 7B/40B, recuperado de: https://lambdalabs.com/blog/finetuning-falcon-llm-7b/40b, 2023spa
dc.relation.references1littlecoder, Web Tutorial: Falcon-7B-Instruct LLM with LangChain Tutorial, Recuperado de: https://www.youtube.com/watch?v=mAoNANPOsd0, 2023spa
dc.relation.referencesAWS, Official Documentation: Instancias P3 de Amazon EC2, Recuperado de: https://aws.amazon.com/es/ec2/instance-types/p3/spa
dc.relation.referencesAbi Aryan, Aakash Kumar Nain, Andrew McMahon, Lucas Augusto Meyer, Harpreet Singh Sahota, The Costly Dilemma: Generalization, Evaluation and Cost-Optimal Deployment of Large Language Models, arXiv:2308.08061, 2023spa
dc.relation.referencesAriel N. Lee, Cole J. Hunter, Nataniel Ruiz, Platypus: Quick, Cheap, and Powerful Refinement of LLMs, arXiv:2308.07317, 2023.spa
dc.relation.referencesE. Almazrouei, H. Alobeidli, A. Alshamsi, A. Cappelli, R. Cojocaru, M. Debbah, E. Goffinet, D. Heslow, J. Launay, Q. Malartic, B. Noune, B. Pannier, and G. Penedo. Falcon-40b: an open large language model with state-of-the-art performance, 2023.spa
dc.relation.referencesH. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M.-A. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023.spa
dc.relation.referencesInstitutional Repository of Universidad Nacional / Trabajos de Grado, recuperado de: https://repositorio.unal.edu.co/handle/unal/5spa
dc.relation.referencesShervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu Richard Socher, Xavier Amatriain, Jianfeng Gao, Large Language Models: A Survey, arXiv: 2402.06196spa
dc.relation.referencesLuis Fernando Niño Vasquez, Laboratorio de investigación en sistemas inteligentes - lisi, Recuperado de: http://www.hermes.unal.edu.co/pages/Consultas/Grupo.xhtml?idGrupo=409&opcion=1spa
dc.relation.referencesRohan Taori and Ishaan Gulrajani and Tianyi Zhang and Yann Dubois and Xuechen Li and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto, Alpaca: A Strong, Replicable Instruction-Following Model, Recuperado de: https://crfm.stanford.edu/2023/03/13/alpaca.html, 2022spa
dc.relation.referencesJulien Launay, TII Falcon LLM License Version 1.0, Recuperado de: https://github.com/DecentralisedAI/falcon-40b/blob/main/LICENSE.txt, 2023spa
dc.relation.referencesMetaAI, Llama License, Recuperado de: https://ai.meta.com/llama/license/, 2023spa
dc.relation.referencesCobus Greyling, Fine-Tuning LLMs With Retrieval Augmented Generation (RAG), Recuperado de: https://cobusgreyling.medium.com/fine-tuning-llms-with-retrieval-augmented-generation-r ag-c66e56aec858spa
dc.relation.referencesMatrixFlows, RAG, Fine-Tuning or Both? A Complete Framework for Choosing the Right Strategy, Recuperado de: https://www.matrixflows.com/blog/retrieval-augmented-generation-rag-finetuning-hybrid-f ramework-for-choosing-right-strategyspa
dc.relation.referencesHarsha Srivatsa, Fine-Tuning versus RAG in Generative AI Applications Architecture, Recuperado de: https://harsha-srivatsa.medium.com/fine-tuning-versus-rag-in-generative-ai-applications-ar chitecture-d54ca6d2acb8spa
dc.relation.referencesZifei Xu, Alexander Lan, Wanzin Yazar, Tristan Webb, Sayeh Sharify, Xin Wang, Scaling Laws for Post Training Quantized Large Language Models, arXiv:2410.12119spa
dc.relation.referencesGraphical Markov models: overview, Nanny Wermuth, D.R. Cox, arXiv:1407.7783, 2014spa
dc.relation.referencesHochreiter Sepp, Schmidhuber Jürgen, Long Short-term Memory, 10.1162/neco.1997.9.8.1735, 1997spa
dc.relation.referencesWenhua Cheng, Weiwei Zhang, Haihao Shen, Yiyang Cai, Xin He, Kaokao Lv, Yi Liu, Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs, arXiv:2309.05516spa
dc.relation.referencesZhihang Yuan, Jiawei Liu, Jiaxiang Wu, Dawei Yang, Qiang Wu, Guangyu Sun, Wenyu Liu, Xinggang Wang, Bingzhe Wu, Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performance, arXiv:2303.13003spa
dc.relation.referencesShengwei Xu, Yuxuan Lu, Grant Schoenebeck, Yuqing Kong, Benchmarking LLMs' Judgments with No Gold Standard, arXiv:2411.07127spa
dc.relation.referencesHongyin Luo & Wei Sun, Addition is All You Need for Energy-Efficient Language Models, arXiv:2410.00907v2spa
dc.relation.referencesAditi Singh, Abul Ehtesham, Saket Kumar, Tala Talaei Khoei, Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG, arXiv:2501.09136spa
dc.relation.referencesMarc Pickett, Jeremy Hartman, Ayan Kumar Bhowmick, Raquib-ul Alam, Aditya Vempaty, Better RAG using Relevant Information Gain, arXiv:2407.12101spa
dc.relation.referencesAlbert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed, Mistral 7B, arXiv:2310.06825spa
dc.relation.referencesApache Software foundation, Apache License, Version 2.0, https://www.apache.org/licenses/LICENSE-2.0spa
dc.relation.referencesOpen Source Initiative, https://opensource.org/license/mitspa
dc.relation.referencesDeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z.F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J.L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R.J. Chen, R.L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S.S. Li et al. (100 additional authors not shown), DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, arXiv:2501.12948spa
dc.relation.referencesAitor Arrieta, Miriam Ugarte, Pablo Valle, José Antonio Parejo, Sergio Segura, o3-mini vs DeepSeek-R1: Which One is Safer?, arXiv:2501.18438spa
dc.relation.referencesAn Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zihan Qiu, Qwen2.5 Technical Report, arXiv:2412.15115spa
dc.relation.referencesIsha Chaudhary, Qian Hu, Manoj Kumar, Morteza Ziyadi, Rahul Gupta, Gagandeep Singh, Quantitative Certification of Bias in Large Language Models, arXiv:2405.18780spa
dc.relation.referencesMulti-task Language Understanding on MMLU, https://paperswithcode.com/sota/multi-task-language-understanding-on-mmluspa
dc.relation.referencesWolfram Ravenwolf, LLM Comparison/Test: 25 SOTA LLMs (including QwQ) through 59 MMLU-Pro CS benchmark runs, https://huggingface.co/blog/wolfram/llm-comparison-test-2024-12-04spa
dc.relation.referencesJiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, Jian Guo, A Survey on LLM-as-a-Judge, arXiv:2411.15594spa
dc.relation.referencesAske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, Thomas Back, Reasoning with Large Language Models, a Survey, arXiv:2407.11511spa
dc.rights.accessrightsinfo:eu-repo/semantics/openAccessspa
dc.rights.licenseReconocimiento 4.0 Internacionalspa
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/spa
dc.subject.ddc000 - Ciencias de la computación, información y obras generales::004 - Procesamiento de datos Ciencia de los computadoresspa
dc.subject.ddc000 - Ciencias de la computación, información y obras generales::001 - Conocimientospa
dc.subject.ddc000 - Ciencias de la computación, información y obras generales::003 - Sistemasspa
dc.subject.lembLENGUAJES NATURALESspa
dc.subject.lembNatural languageseng
dc.subject.lembLENGUAJES DE MAQUINAspa
dc.subject.lembProgramming languageseng
dc.subject.lembLENGUAJES DE PROGRAMACION (COMPUTADORES ELECTRONICOS)spa
dc.subject.lembProgramming languages (electronic computers)eng
dc.subject.lembPROCESAMIENTO ELECTRONICO DE DATOSspa
dc.subject.lembElectronic data processingeng
dc.subject.lembLINGUISTICA COMPUTACIONALspa
dc.subject.lembComputational linguisticseng
dc.subject.lembLEXICOGRAFIA-PROCESAMIENTO DE DATOSspa
dc.subject.lembLexicography Data processingeng
dc.subject.lembAPRENDIZAJE AUTOMATICO (INTELIGENCIA ARTIFICIAL)spa
dc.subject.lembMachine learningeng
dc.subject.lembINTELIGENCIA ARTIFICIAL-PROCESAMIENTO DE DATOSspa
dc.subject.lembArtificial intelligen - data processingeng
dc.subject.proposalGrandes Modelos de Lenguaje (LLMs)spa
dc.subject.proposalEficiencia computacionalspa
dc.subject.proposalEntrenamiento eficientespa
dc.subject.proposalBenchmarks de Modelos de Lenguajespa
dc.subject.proposalLarge Language Models (LLMs)eng
dc.subject.proposalComputational Efficiencyeng
dc.subject.proposalEfficient Trainingeng
dc.subject.proposalLanguage Model Benchmarkseng
dc.subject.wikidataSemantic Webeng
dc.subject.wikidataWeb semánticaspa
dc.titleEstrategia eficiente para la mejora de las capacidades de modelos grandes de lenguaje (LLMs)spa
dc.title.translatedEfficient strategy for improving the capabilities of large language models (LLMs)eng
dc.typeTrabajo de grado - Maestríaspa
dc.type.coarhttp://purl.org/coar/resource_type/c_bdccspa
dc.type.coarversionhttp://purl.org/coar/version/c_ab4af688f83e57aaspa
dc.type.contentTextspa
dc.type.driverinfo:eu-repo/semantics/masterThesisspa
dc.type.redcolhttp://purl.org/redcol/resource_type/TMspa
dc.type.versioninfo:eu-repo/semantics/acceptedVersionspa
dcterms.audience.professionaldevelopmentEstudiantesspa
oaire.accessrightshttp://purl.org/coar/access_right/c_abf2spa
oaire.fundernameJulián Camilo Velandia Gutiérrezspa

Archivos

Bloque original

Mostrando 1 - 1 de 1
Cargando...
Miniatura
Nombre:
DocumentoFinal1026598964 (1).pdf
Tamaño:
3.84 MB
Formato:
Adobe Portable Document Format
Descripción:
Tesis de Maestría en Ingeniería de Sistemas y Computación

Bloque de licencias

Mostrando 1 - 1 de 1
Cargando...
Miniatura
Nombre:
license.txt
Tamaño:
5.74 KB
Formato:
Item-specific license agreed upon to submission
Descripción: