Paper Digest: Recent Papers on Transformer
Paper Digest Team extracted all recent Transformer (NLP) related papers on our radar, and generated highlight sentences for them. The results are then sorted by relevance & date. In addition to this ‘static’ page, we also provide a real-time version of this article, which has more coverage and is updated in real time to include the most recent updates on this topic.
This list is created by the Paper Digest Team. Experience the cutting-edge capabilities of Paper Digest, an innovative AI-powered research platform that empowers you to write, review, get answers and more. Try us today and unlock the full potential of our services for free!
TABLE 1: Paper Digest: Recent Papers on Transformer
| Paper | Author(s) | Source | Date | |
|---|---|---|---|---|
| 1 | Grokked Transformers Are Implicit Reasoners: A Mechanistic Journey to The Edge of Generalization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We study whether transformers can learn to *implicitly* reason over parametric knowledge, a skill that even the most capable language models struggle with. |
Boshi Wang; Xiang Yue; Yu Su; Huan Sun; | nips | 2024-10-07 |
| 2 | Are Self-Attentions Effective for Time Series Forecasting? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we shift focus from the overall architecture of the Transformer to the effectiveness of self-attentions for time series forecasting. |
Dongbin Kim; Jinseong Park; Jaewook Lee; Hoki Kim; | nips | 2024-10-07 |
| 3 | Finding Transformer Circuits With Edge Pruning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we frame circuit discovery as an optimization problem and propose _Edge Pruning_ as an effective and scalable solution. |
Adithya Bhaskar; Alexander Wettig; Dan Friedman; Danqi Chen; | nips | 2024-10-07 |
| 4 | Permutation-Invariant Autoregressive Diffusion for Graph Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce PARD, a Permutation-invariant Auto Regressive Diffusion model that integrates diffusion models with autoregressive methods. |
Lingxiao Zhao; Xueying Ding; Leman Akoglu; | nips | 2024-10-07 |
| 5 | In-Context Learning State Vector with Inner and Momentum Optimization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, the working mechanisms and optimization of these vectors are yet to be thoroughly explored. In this paper, we address this gap by presenting a comprehensive analysis of these compressed vectors, drawing parallels to the parameters trained with gradient descent, and introducing the concept of state vector. |
DONGFANG LI et. al. | nips | 2024-10-07 |
| 6 | CulturePark: Boosting Cross-cultural Understanding in Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Inspired by cognitive theories on social communication, this paper introduces CulturePark, an LLM-powered multi-agent communication framework for cultural data collection. |
CHENG LI et. al. | nips | 2024-10-07 |
| 7 | VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision Computation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we introduce a novel approach to reduce vision compute by leveraging redundant vision tokens “skipping layers” rather than decreasing the number of vision tokens. |
SHIWEI WU et. al. | nips | 2024-10-07 |
| 8 | Timer-XL: Long-Context Transformers for Unified Time Series Forecasting Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present Timer-XL, a generative Transformer for unified time series forecasting. |
Yong Liu; Guo Qin; Xiangdong Huang; Jianmin Wang; Mingsheng Long; | arxiv-cs.LG | 2024-10-07 |
| 9 | Learning to Grok: Emergence of In-context Learning and Skill Composition in Modular Arithmetic Tasks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we study the emergence of in-context learning and skill composition in a collection of modular arithmetic tasks. |
Tianyu He; Darshil Doshi; Aritra Das; Andrey Gromov; | nips | 2024-10-07 |
| 10 | RL-GPT: Integrating Reinforcement Learning and Code-as-policy Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To seamlessly integrate both modalities, we introduce a two-level hierarchical framework, RL-GPT, comprising a slow agent and a fast agent. |
SHAOTENG LIU et. al. | nips | 2024-10-07 |
| 11 | JiuZhang3.0: Efficiently Improving Mathematical Reasoning By Training Small Data Synthesis Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To reduce the cost, based on open-source available texts, we propose an efficient way that trains a small LLM for math problem synthesis, to efficiently generate sufficient high-quality pre-training data. |
KUN ZHOU et. al. | nips | 2024-10-07 |
| 12 | ChatQA: Surpassing GPT-4 on Conversational QA and RAG Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we introduce ChatQA, a suite of models that outperform GPT-4 on retrieval-augmented generation (RAG) and conversational question answering (QA). |
ZIHAN LIU et. al. | nips | 2024-10-07 |
| 13 | Achieving Efficient Alignment Through Learned Correction Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we introduce *Aligner*, a novel and simple alignment paradigm that learns the correctional residuals between preferred and dispreferred answers using a small model. |
JIAMING JI et. al. | nips | 2024-10-07 |
| 14 | Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we exploit advancements in Multimodal Large Language Models (MLLMs), particularly GPT-4V, to present a novel approach, Make-it-Real: 1) We demonstrate that GPT-4V can effectively recognize and describe materials, allowing the construction of a detailed material library. |
YE FANG et. al. | nips | 2024-10-07 |
| 15 | APIGen: Automated PIpeline for Generating Verifiable and Diverse Function-Calling Datasets Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents APIGen, an automated data generation pipeline designed to produce verifiable high-quality datasets for function-calling applications. |
ZUXIN LIU et. al. | nips | 2024-10-07 |
| 16 | DeformableTST: Transformer for Time Series Forecasting Without Over-reliance on Patching Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: But at the same time, we observe a new problem that the recent Transformer-based models are overly reliant on patching to achieve ideal performance, which limits their applicability to some forecasting tasks unsuitable for patching. In this paper, we intent to handle this emerging issue. |
Donghao Luo; Xue Wang; | nips | 2024-10-07 |
| 17 | HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Identifying optimal model configurations under specific hardware constraints is becoming essential but remains challenging due to the computational load of exhaustive training and evaluation on multiple devices. To address this, we introduce HW-GPT-Bench, a hardware-aware benchmark that utilizes surrogate predictions to approximate various hardware metrics across 13 devices of architectures in the GPT-2 family, with architectures containing up to 774M parameters. |
RHEA SUKTHANKER et. al. | nips | 2024-10-07 |
| 18 | Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learner Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we propose Efficient Multi-Task Learning (EMTAL), a novel approach that transforms a pre-trained Transformer into an efficient multi-task learner during training, and reparameterizes the knowledge back to the original Transformer for efficient inference. |
Hanwen Zhong; Jiaxin Chen; Yutong Zhang; Di Huang; Yunhong Wang; | nips | 2024-10-07 |
| 19 | SAND: Smooth Imputation of Sparse and Noisy Functional Data with Transformer Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Although the transformer architecture has come to dominate other models for text and image data, its application to irregularly-spaced longitudinal data has been limited. We introduce a variant of the transformer that enables it to more smoothly impute such functional data. |
Ju-Sheng Hong; Junwen Yao; Jonas Mueller; Jane-Ling Wang; | nips | 2024-10-07 |
| 20 | InversionView: A General-Purpose Method for Reading Information from Neural Activations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The inner workings of neural networks can be better understood if we can fully decipher the information encoded in neural activations. In this paper, we argue that this information is embodied by the subset of inputs that give rise to similar activations. |
Xinting Huang; Madhur Panwar; Navin Goyal; Michael Hahn; | nips | 2024-10-07 |
| 21 | ETO:Efficient Transformer-based Local Feature Matching By Organizing Multiple Homography Hypotheses Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose an efficient transformer-based network architecture for local feature matching. |
JUNJIE NI et. al. | nips | 2024-10-07 |
| 22 | A Critical Evaluation of AI Feedback for Aligning Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We show that the improvements of the RL step are virtually entirely due to the widespread practice of using a weaker teacher model (e.g. GPT-3.5) for SFT data collection than the critic (e.g., GPT-4) used for AI feedback generation. |
ARCHIT SHARMA et. al. | nips | 2024-10-07 |
| 23 | LSH-MoE: Communication-efficient MoE Training Via Locality-Sensitive Hashing Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose LSH-MoE, a communication-efficient MoE training framework using locality-sensitive hashing (LSH). |
XIAONAN NIE et. al. | nips | 2024-10-07 |
| 24 | EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The former hurts the fairness of benchmarks, and the latter hinders practitioners from selecting superior LLMs for specific programming domains. To address these two limitations, we propose a new benchmark – EvoCodeBench, which has the following advances: (1) Evolving data. |
JIA LI et. al. | nips | 2024-10-07 |
| 25 | Accelerating Transformers with Spectrum-Preserving Token Merging Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Prior work has proposed algorithms based on Bipartite Soft Matching (BSM), which divides tokens into distinct sets and merges the top $k$ similar tokens. |
CHAU TRAN et. al. | nips | 2024-10-07 |
| 26 | SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce SELF-DISCOVER, a general framework for LLMs to self-discover the task-intrinsic reasoning structures to tackle complex reasoning problems that are challenging for typical prompting methods. |
PEI ZHOU et. al. | nips | 2024-10-07 |
| 27 | Limits of Transformer Language Models on Learning to Compose Algorithms Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To this end, we evaluate training LLaMA models and prompting GPT-4 and Gemini on four tasks demanding to learn a composition of several discrete sub-tasks. |
JONATHAN THOMM et. al. | nips | 2024-10-07 |
| 28 | The Mamba in The Llama: Distilling and Accelerating Hybrid Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Recent research suggests that state-space models (SSMs) like Mamba can be competitive with Transformer models for language modeling with advantageous deployment characteristics. Given the focus and expertise on training large-scale Transformer models, we consider the challenge of converting these pretrained models into SSMs for deployment. |
Junxiong Wang; Daniele Paliotta; Avner May; Alexander Rush; Tri Dao; | nips | 2024-10-07 |
| 29 | Divide-and-Conquer Meets Consensus: Unleashing The Power of Functions in Code Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To this end, we propose FunCoder, a code generation framework incorporating the divide-and-conquer strategy with functional consensus. |
JINGCHANG CHEN et. al. | nips | 2024-10-07 |
| 30 | SemCoder: Training Code Language Models with Comprehensive Semantics Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper aims to bridge the gap between Code LLMs’ reliance on static text data and the need for thorough semantic understanding for complex tasks like debugging and program repair. |
YANGRUIBO DING et. al. | nips | 2024-10-07 |
| 31 | ProTransformer: Robustify Transformers Via Plug-and-Play Paradigm Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we introduce a novel robust attention mechanism designed to enhance the resilience of transformer-based architectures. |
Zhichao Hou; Weizhi Gao; Yuchen Shen; Feiyi Wang; Xiaorui Liu; | nips | 2024-10-07 |
| 32 | Rethinking Decoders for Transformer-based Semantic Segmentation: Compression Is All You Need Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we attempt to derive their white-box counterparts from the perspective of compression. |
Qishuai Wen; Chun-Guang Li; | nips | 2024-10-07 |
| 33 | Weak-to-Strong Search: Align Large Language Models Via Searching Over Small Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we introduce *weak-to-strong search*, framing the alignment of a large language model as a test-time greedy search to maximize the log-likelihood difference between small tuned and untuned models while sampling from the frozen large model. |
ZHANHUI ZHOU et. al. | nips | 2024-10-07 |
| 34 | Unraveling The Gradient Descent Dynamics of Transformers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: While the Transformer architecture has achieved remarkable success across various domains, a thorough theoretical foundation explaining its optimization dynamics is yet to be fully developed. In this study, we aim to bridge this understanding gap by answering the following two core questions: (1) Which types of Transformer architectures allow Gradient Descent (GD) to achieve guaranteed convergence? |
Bingqing Song; Boran Han; Shuai Zhang; Jie Ding; Mingyi Hong; | nips | 2024-10-07 |
| 35 | LLaVA Needs More Knowledge: Retrieval Augmented Natural Language Generation with Knowledge Graph for Explaining Thoracic Pathologies Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The KG datastore is designed as a plug-and-play module, allowing for seamless integration with various model architectures. We introduce and evaluate three distinct frameworks within this paradigm: KG-LLaVA, which integrates the pre-trained LLaVA model with KG-RAG; Med-XPT, a custom framework combining MedCLIP, a transformer-based projector, and GPT-2; and Bio-LLaVA, which adapts LLaVA by incorporating the Bio-ViT-L vision model. |
Ameer Hamza; Yong Hyun Ahn; Sungyoung Lee; Seong Tae Kim; | arxiv-cs.CV | 2024-10-07 |
| 36 | OccamLLM: Fast and Exact Language Model Arithmetic in A Single Step Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a framework that enables exact arithmetic in a single autoregressive step, providing faster, more secure, and more interpretable LLM systems with arithmetic capabilities. |
OWEN DUGAN et. al. | nips | 2024-10-07 |
| 37 | Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we present a method that is able to distill a pre-trained Transformer architecture into alternative architectures such as state space models (SSMs). |
Aviv Bick; Kevin Li; Eric Xing; J. Zico Kolter; Albert Gu; | nips | 2024-10-07 |
| 38 | Stabilize The Latent Space for Image Autoregressive Modeling: A Unified Perspective Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This finding contrasts sharply with the field of NLP, where the autoregressive model GPT has established a commanding presence. To address this discrepancy, we introduce a unified perspective on the relationship between latent space and generative models, emphasizing the stability of latent space in image generative modeling. |
YONGXIN ZHU et. al. | nips | 2024-10-07 |
| 39 | U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Based on self-attention with downsampled tokens, we propose a series of U-shaped DiTs (U-DiTs) in the paper and conduct extensive experiments to demonstrate the extraordinary performance of U-DiT models. |
YUCHUAN TIAN et. al. | nips | 2024-10-07 |
| 40 | InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Specifically, this research advances the patch division paradigm by introducing a novel extension: dynamic resolution with automatic patch configuration. |
XIAOYI DONG et. al. | nips | 2024-10-07 |
| 41 | SpikedAttention: Training-Free and Fully Spike-Driven Transformer-to-SNN Conversion with Winner-Oriented Spike Shift for Softmax Operation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we propose a novel transformer-to-SNN conversion method that outputs an end-to-end spike-based transformer, named SpikedAttention. |
Sangwoo Hwang; Seunghyun Lee; Dahoon Park; Donghun Lee; Jaeha Kung; | nips | 2024-10-07 |
| 42 | Understanding Transformers Via N-Gram Statistics Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper takes a first step in this direction by considering families of functions (i.e. rules) formed out of simple N-gram based statistics of the training data. By studying how well these rulesets approximate transformer predictions, we obtain a variety of novel discoveries: a simple method to detect overfitting during training without using a holdout set, a quantative measure of how transformers progress from learning simple to more complex statistical rules over the course of training, a model-variance criterion governing when transformer predictions tend to be described by N-gram rules, and insights into how well transformers can be approximated by N-gram rulesets in the limit where these rulesets become increasingly complex. |
Timothy Nguyen; | nips | 2024-10-07 |
| 43 | FinBen: An Holistic Financial Benchmark for Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we introduce FinBen, the first extensive open-source evaluation benchmark, including 36 datasets spanning 24 financial tasks, covering seven critical aspects: information extraction (IE), textual analysis, question answering (QA), text generation, risk management, forecasting, and decision-making. |
QIANQIAN XIE et. al. | nips | 2024-10-07 |
| 44 | Alleviating Distortion in Image Generation Via Multi-Resolution Diffusion Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents innovative enhancements to diffusion models by integrating a novel multi-resolution network and time-dependent layer normalization. |
QIHAO LIU et. al. | nips | 2024-10-07 |
| 45 | DenseFormer: Enhancing Information Flow in Transformers Via Depth Weighted Averaging Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The transformer architecture by Vaswani et al. (2017) is now ubiquitous across application domains, from natural language processing to speech processing and image understanding. We propose DenseFormer, a simple modification to the standard architecture that improves the perplexity of the model without increasing its size—adding a few thousand parameters for large-scale models in the 100B parameters range. |
Matteo Pagliardini; Amirkeivan Mohtashami; François Fleuret; Martin Jaggi; | nips | 2024-10-07 |
| 46 | Approximation Rate of The Transformer Architecture for Sequence Modeling Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we investigate the approximation rate for single-layer Transformers with one head. |
Haotian Jiang; Qianxiao Li; | nips | 2024-10-07 |
| 47 | Understanding The Expressive Power and Mechanisms of Transformer for Sequence Modeling Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We conduct a systematic study of the approximation properties of Transformer for sequence modeling with long, sparse and complicated memory. |
Mingze Wang; Weinan E; | nips | 2024-10-07 |
| 48 | MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Motivated by the empirical findings, we propose a novel LLM-based **M**ulti-**A**gent framework for **G**itHub **I**ssue re**S**olution, **MAGIS**, consisting of four agents customized for software evolution: Manager, Repository Custodian, Developer, and Quality Assurance Engineer agents. |
WEI TAO et. al. | nips | 2024-10-07 |
| 49 | $M^3$GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents $M^3$GPT, an advanced \textbf{M}ultimodal, \textbf{M}ultitask framework for \textbf{M}otion comprehension and generation. |
MINGSHUANG LUO et. al. | nips | 2024-10-07 |
| 50 | Differential Transformer Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we introduce Diff Transformer, which amplifies attention to the relevant context while canceling noise. |
TIANZHU YE et. al. | arxiv-cs.CL | 2024-10-07 |
| 51 | Perception of Knowledge Boundary for Large Language Models Through Semi-open-ended Question Answering Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we perceive the LLMs’ knowledge boundary with semi-open-ended questions by discovering more ambiguous answers. |
ZHIHUA WEN et. al. | nips | 2024-10-07 |
| 52 | Can Large Language Models Explore In-context? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We investigate the extent to which contemporary Large Language Models (LLMs) can engage in exploration, a core capability in reinforcement learning and decision making. |
Akshay Krishnamurthy; Keegan Harris; Dylan J Foster; Cyril Zhang; Aleksandrs Slivkins; | nips | 2024-10-07 |
| 53 | Predicting Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Yet, despite sustained widespread interest, a rigorous understanding of why transformer scaling laws exist is still missing. To answer this question, we establish novel statistical estimation and mathematical approximation theories for transformers when the input data are concentrated on a low-dimensional manifold. |
Alexander Havrilla; Wenjing Liao; | nips | 2024-10-07 |
| 54 | UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Construction Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper presents UrbanKGent, a unified large language model agent framework, for urban knowledge graph construction. |
Yansong Ning; Hao Liu; | nips | 2024-10-07 |
| 55 | Visual Autoregressive Modeling: Scalable Image Generation Via Next-Scale Prediction Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present Visual AutoRegressive modeling (VAR), a new generation paradigm that redefines the autoregressive learning on images as coarse-to-fine next-scale prediction or next-resolution prediction, diverging from the standard raster-scan next-token prediction. |
Keyu Tian; Yi Jiang; Zehuan Yuan; BINGYUE PENG; Liwei Wang; | nips | 2024-10-07 |
| 56 | Seshat Global History Databank Text Dataset and Benchmark of Large Language Models’ History Knowledge Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This benchmarking is particularly challenging, given that human knowledge of history is inherently unbalanced, with more information available on Western history and recent periods. To address this challenge, we introduce a curated sample of the Seshat Global History Databank, which provides a structured representation of human historical knowledge, containing 36,000 data points across 600 historical societies and over 600 scholarly references. |
JAKOB HAUSER et. al. | nips | 2024-10-07 |
| 57 | Query-Based Adversarial Prompt Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We improve on prior work with a query-based attack that leverages API access to a remote language model to construct adversarial examples that cause the model to emit harmful strings with (much) higher probability than with transfer-only attacks. |
Jonathan Hayase; Ema Borevković; Nicholas Carlini; Florian Tramer; Milad Nasr; | nips | 2024-10-07 |
| 58 | DAMRO: Dive Into The Attention Mechanism of LVLM to Reduce Object Hallucination Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To address the issue, we propose DAMRO, a novel training-free strategy that $D$ive into $A$ttention $M$echanism of LVLM to $R$educe $O$bject Hallucination. |
Xuan Gong; Tianshi Ming; Xinpeng Wang; Zhihua Wei; | arxiv-cs.CL | 2024-10-06 |
| 59 | ProtocoLLM: Automatic Evaluation Framework of LLMs on Domain-Specific Scientific Protocol Formulation Tasks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Here, we propose a flexible, automatic framework to evaluate LLM’s capability on SPFT: ProtocoLLM. |
Seungjun Yi; Jaeyoung Lim; Juyong Yoon; | arxiv-cs.CL | 2024-10-06 |
| 60 | Fundamental Limitations on Subquadratic Alternatives to Transformers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: For instance, state space models such as Mamba were designed to replace attention with an almost linear time alternative. In this paper, we prove that any such approach cannot perform important tasks that Transformer is able to perform (assuming a popular conjecture from fine-grained complexity theory). |
Josh Alman; Hantao Yu; | arxiv-cs.LG | 2024-10-05 |
| 61 | Equivariant Neural Functional Networks for Transformers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: While NFN have been extensively developed for MLP and CNN, no prior work has addressed their design for transformers, despite the importance of transformers in modern deep learning. This paper aims to address this gap by providing a systematic study of NFN for transformers. |
VIET-HOANG TRAN et. al. | arxiv-cs.LG | 2024-10-05 |
| 62 | Still Not Quite There! Evaluating Large Language Models for Comorbid Mental Health Diagnosis Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, we introduce ANGST, a novel, first-of-its kind benchmark for depression-anxiety comorbidity classification from social media posts. |
AMEY HENGLE et. al. | arxiv-cs.CL | 2024-10-04 |
| 63 | Selective Transformer for Hyperspectral Image Classification Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, existing Transformer models face two key challenges when dealing with HSI scenes characterized by diverse land cover types and rich spectral information: (1) fixed receptive field representation overlooks effective contextual information; (2) redundant self-attention feature representation. To address these limitations, we propose a novel Selective Transformer (SFormer) for HSI classification. |
Yichu Xu; Di Wang; Lefei Zhang; Liangpei Zhang; | arxiv-cs.CV | 2024-10-04 |
| 64 | How Language Models Prioritize Contextual Grammatical Cues? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we investigate how language models handle gender agreement when multiple gender cue words are present, each capable of independently disambiguating a target gender pronoun. |
Hamidreza Amirzadeh; Afra Alishahi; Hosein Mohebbi; | arxiv-cs.CL | 2024-10-04 |
| 65 | Learning Semantic Structure Through First-Order-Logic Translation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we study whether transformer-based language models can extract predicate argument structure from simple sentences. |
Akshay Chaturvedi; Nicholas Asher; | arxiv-cs.CL | 2024-10-04 |
| 66 | Dynamic Diffusion Transformer Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Specifically, we introduce a Timestep-wise Dynamic Width (TDW) approach that adapts model width conditioned on the generation timesteps. |
WANGBO ZHAO et. al. | arxiv-cs.CV | 2024-10-04 |
| 67 | Towards Linguistically-Aware and Language-Independent Tokenization for Large Language Models (LLMs) Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents a comprehensive study on the tokenization techniques employed by state-of-the-art large language models (LLMs) and their implications on the cost and availability of services across different languages, especially low resource languages. |
Abrar Rahman; Garry Bowlin; Binit Mohanty; Sean McGunigal; | arxiv-cs.CL | 2024-10-04 |
| 68 | AlphaIntegrator: Transformer Action Search for Symbolic Integration Proofs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present the first correct-by-construction learning-based system for step-by-step mathematical integration. |
Mert Ünsal; Timon Gehr; Martin Vechev; | arxiv-cs.LG | 2024-10-03 |
| 69 | CulturalBench: A Robust, Diverse and Challenging Benchmark on Measuring The (Lack Of) Cultural Knowledge of LLMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce CulturalBench: a set of 1,227 human-written and human-verified questions for effectively assessing LLMs’ cultural knowledge, covering 45 global regions including the underrepresented ones like Bangladesh, Zimbabwe, and Peru. |
YU YING CHIU et. al. | arxiv-cs.CL | 2024-10-03 |
| 70 | SIEVE: General Purpose Data Filtering System Matching GPT-4o Accuracy at 1% The Cost Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper proposes SIEVE, a lightweight alternative that matches GPT-4o accuracy at a fraction of the cost. |
Jifan Zhang; Robert Nowak; | arxiv-cs.CL | 2024-10-03 |
| 71 | IndicSentEval: How Effectively Do Multilingual Transformer Models Encode Linguistic Properties for Indic Languages? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we investigate similar questions regarding encoding capability and robustness for 8 linguistic properties across 13 different perturbations in 6 Indic languages, using 9 multilingual Transformer models (7 universal and 2 Indic-specific). |
Akhilesh Aravapalli; Mounika Marreddy; Subba Reddy Oota; Radhika Mamidi; Manish Gupta; | arxiv-cs.CL | 2024-10-03 |
| 72 | Intrinsic Evaluation of RAG Systems for Deep-Logic Questions Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce the Overall Performance Index (OPI), an intrinsic metric to evaluate retrieval-augmented generation (RAG) mechanisms for applications involving deep-logic queries. |
Junyi Hu; You Zhou; Jie Wang; | arxiv-cs.AI | 2024-10-03 |
| 73 | Coal Mining Question Answering with LLMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we present a novel approach to coal mining question answering (QA) using large language models (LLMs) combined with tailored prompt engineering techniques. |
Antonio Carlos Rivera; Anthony Moore; Steven Robinson; | arxiv-cs.CL | 2024-10-03 |
| 74 | Automatic Deductive Coding in Discourse Analysis: An Application of Large Language Models in Learning Analytics Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To evaluate the usefulness of large language models in automatic deductive coding, we employed three different classification methods driven by different artificial intelligence technologies, including the traditional text classification method with text feature engineering, BERT-like pretrained language model and GPT-like pretrained large language model (LLM). We applied these methods to two different datasets and explored the potential of GPT and prompt engineering in automatic deductive coding. |
Lishan Zhang; Han Wu; Xiaoshan Huang; Tengfei Duan; Hanxiang Du; | arxiv-cs.CL | 2024-10-02 |
| 75 | Emotion-Aware Response Generation Using Affect-Enriched Embeddings with LLMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce a novel framework that integrates multiple emotion lexicons, including NRC Emotion Lexicon, VADER, WordNet, and SentiWordNet, with state-of-the-art LLMs such as LLAMA 2, Flan-T5, ChatGPT 3.0, and ChatGPT 4.0. |
Abdur Rasool; Muhammad Irfan Shahzad; Hafsa Aslam; Vincent Chan; | arxiv-cs.CL | 2024-10-02 |
| 76 | A Spark of Vision-Language Intelligence: 2-Dimensional Autoregressive Transformer for Efficient Finegrained Image Generation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This work tackles the information loss bottleneck of vector-quantization (VQ) autoregressive image generation by introducing a novel model architecture called the 2-Dimensional Autoregression (DnD) Transformer. |
LIANG CHEN et. al. | arxiv-cs.CV | 2024-10-02 |
| 77 | Improving Autonomous AI Agents with Reflective Tree Search and Self-Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, even state-of-the-art vision-language models (VLMs), such as GPT-4o, still fall short of human-level performance, particularly in intricate web environments and long-horizon planning tasks. To address these limitations, we introduce Reflective Monte Carlo Tree Search (R-MCTS), a novel test-time algorithm designed to enhance the ability of AI agents, e.g., powered by GPT-4o, to explore decision space on the fly. |
XIAO YU et. al. | arxiv-cs.CL | 2024-10-02 |
| 78 | On The Adaptation of Unlimiformer for Decoder-Only Transformers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, its main limitation is incompatibility with decoder-only transformers out of the box. In this work, we explore practical considerations of adapting Unlimiformer to decoder-only transformers and introduce a series of modifications to overcome this limitation. |
KIAN AHRABIAN et. al. | arxiv-cs.CL | 2024-10-02 |
| 79 | Financial Sentiment Analysis on News and Reports Using Large Language Models and FinBERT Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper investigates the application of LLMs and FinBERT for FSA, comparing their performance on news articles, financial reports and company announcements. |
Yanxin Shen; Pulin Kirin Zhang; | arxiv-cs.IR | 2024-10-02 |
| 80 | SIGMA: Secure GPT Inference with Function Secret Sharing IF:3 Summary Related Papers Related Patents Related Grants Related Venues Related Experts View Abstract: Secure 2-party computation (2PC) enables secure inference that offers protection for both proprietary machine learning (ML) models and sensitive inputs to them. However, the … |
KANAV GUPTA et. al. | Proc. Priv. Enhancing Technol. | 2024-10-01 |
| 81 | Creative and Context-Aware Translation of East Asian Idioms with GPT-4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, compiling a dictionary of candidate translations demands much time and creativity even for expert translators. To alleviate such burden, we evaluate if GPT-4 can help generate high-quality translations. |
Kenan Tang; Peiyang Song; Yao Qin; Xifeng Yan; | arxiv-cs.CL | 2024-10-01 |
| 82 | MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone’s Potential with Masked Autoregressive Pretraining Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Based on this analysis, we propose Masked Autoregressive Pretraining (MAP) to pretrain a hybrid Mamba-Transformer vision backbone network. |
Yunze Liu; Li Yi; | arxiv-cs.CV | 2024-10-01 |
| 83 | LingoQA: Video Question Answering for Autonomous Driving Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce LingoQA, a novel dataset and benchmark for visual question answering in autonomous driving.We release our dataset and benchmark1 as an evaluation platform for vision-language models in autonomous driving. |
ANA-MARIA MARCU et. al. | eccv | 2024-09-30 |
| 84 | TaskComplexity: A Dataset for Task Complexity Classification with In-Context Learning, FLAN-T5 and GPT-4o Benchmarks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper addresses the challenge of classifying and assigning programming tasks to experts, a process that typically requires significant effort, time, and cost. |
Areeg Fahad Rasheed; M. Zarkoosh; Safa F. Abbas; Sana Sabah Al-Azzawi; | arxiv-cs.CL | 2024-09-30 |
| 85 | GiT: Towards Generalist Vision Transformer Through Universal Language Interface Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper proposes a simple, yet effective framework, called , simultaneously applicable for various vision tasks only with a vanilla ViT.Interestingly, our builds a new benchmark in generalist performance, and fosters mutual enhancement across tasks, leading to significant improvements compared to isolated training. |
HAIYANG WANG et. al. | eccv | 2024-09-30 |
| 86 | An Explainable Vision Question Answer Model Via Diffusion Chain-of-Thought Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This means that generating explanations solely for the answer can lead to a semantic discrepancy between the content of the explanation and the question-answering content. To address this, we propose a step-by-step reasoning approach to reduce such semantic discrepancies. |
Chunhao LU; Qiang Lu; Jake Luo; | eccv | 2024-09-30 |
| 87 | GENIXER: Empowering Multimodal Large Language Models As A Powerful Data Generator Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce , a comprehensive data generation pipeline consisting of four key steps: (i) instruction data collection, (ii) instruction template design, (iii) empowering MLLMs, and (iv) data generation and filtering. |
Henry Hengyuan Zhao; Pan Zhou; Mike Zheng Shou; | eccv | 2024-09-30 |
| 88 | HyTAS: A Hyperspectral Image Transformer Architecture Search Benchmark and Analysis Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Transformer architectures have significantly enhanced HSI task performance, while advancements in Transformer Architecture Search (TAS) have improved model discovery. To harness these advancements for HSI classification, we make the following contributions: i) We propose HyTAS, the first benchmark on transformer architecture search for Hyperspectral imaging, ii) We comprehensively evaluate 12 different methods to identify the optimal transformer over 5 different datasets, iii) We perform an extensive factor analysis on the Hyperspectral transformer search performance, greatly motivating future research in this direction. |
Fangqin Zhou; Mert Kilickaya; Joaquin Vanschoren; Ran Piao; | eccv | 2024-09-30 |
| 89 | Comprehensive Performance Modeling and System Design Insights for Foundation Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We analyze performance characteristics of such transformer models and discuss their sensitivity to the transformer type, parallelization strategy, and HPC system features (accelerators and interconnects). We utilize a performance model that allows us to explore this complex design space and highlight its key components. |
SHASHANK SUBRAMANIAN et. al. | arxiv-cs.LG | 2024-09-30 |
| 90 | MaskMamba: A Hybrid Mamba-Transformer Model for Masked Image Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, we introduce MaskMamba, a novel hybrid model that combines Mamba and Transformer architectures, utilizing Masked Image Modeling for non-autoregressive image synthesis. |
Wenchao Chen; Liqiang Niu; Ziyao Lu; Fandong Meng; Jie Zhou; | arxiv-cs.CV | 2024-09-30 |
| 91 | An Efficient and Effective Transformer Decoder-Based Framework for Multi-Task Visual Grounding Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This quadratic increase in computational burden restricts the applicability of visual grounding to more intricate scenes, such as conversation-based reasoning segmentation, which involves lengthy language expressions. In this paper, we propose an efficient and effective multi-task visual grounding (EEVG) framework based on Transformer Decoder to address this issue, which reduces the cost in both language and visual aspects. |
Wei Chen; Long Chen; Yu Wu; | eccv | 2024-09-30 |
| 92 | OccWorld: Learning A 3D Occupancy World Model for Autonomous Driving Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we explore a new framework of learning a world model, OccWorld, in the 3D occupancy space to simultaneously predict the movement of the ego car and the evolution of the surrounding scenes. |
WENZHAO ZHENG et. al. | eccv | 2024-09-30 |
| 93 | MART: MultiscAle Relational Transformer Networks for Multi-agent Trajectory Prediction Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, the hypergraph transformer-based method for trajectory prediction is yet to be explored. Therefore, we present a MultiscAle Relational Transformer (MART) network for multi-agent trajectory prediction. |
Seongju Lee; Junseok Lee; Yeonguk Yu; Taeri Kim; Kyoobin Lee; | eccv | 2024-09-30 |
| 94 | Evaluating The Fairness of Task-adaptive Pretraining on Unlabeled Test Data Before Few-shot Text Classification Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Few-shot learning benchmarks are critical for evaluating modern NLP techniques. |
Kush Dubey; | arxiv-cs.CL | 2024-09-30 |
| 95 | Analysis-by-Synthesis Transformer for Single-View 3D Reconstruction Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, existing methods face limitations in both shape reconstruction and texture generation. This paper introduces an innovative Analysis-by-Synthesis Transformer that addresses these limitations in a unified framework by effectively modeling pixel-to-shape and pixel-to-texture relationships. |
DIAN JIA et. al. | eccv | 2024-09-30 |
| 96 | Sparse Attention Decomposition Applied to Circuit Tracing Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work we seek to isolate and identify the features used to effect communication and coordination among attention heads in GPT-2 small. |
Gabriel Franco; Mark Crovella; | arxiv-cs.LG | 2024-09-30 |
| 97 | Depression Detection in Social Media Posts Using Transformer-based Models and Auxiliary Features Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Existing studies have explored various approaches to this problem but often fall short in terms of accuracy and robustness. To address these limitations, this research proposes a neural network architecture leveraging transformer-based models combined with metadata and linguistic markers. |
Marios Kerasiotis; Loukas Ilias; Dimitris Askounis; | arxiv-cs.CL | 2024-09-30 |
| 98 | AutoEval-Video: An Automatic Benchmark for Assessing Large Vision Language Models in Open-Ended Video Question Answering Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a novel and challenging benchmark, AutoEval-Video, to comprehensively evaluate large vision-language models in open-ended video question answering. |
Weiran Huang; Xiuyuan Chen; Yuan Lin; Yuchen Zhang; | eccv | 2024-09-30 |
| 99 | Optimizing Factorized Encoder Models: Time and Memory Reduction for Scalable and Efficient Action Recognition Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we address the challenges posed by the substantial training time and memory consumption associated with video transformers, focusing on the ViViT (Video Vision Transformer) model, in particular the Factorised Encoder version, as our baseline for action recognition tasks. |
Shreyank N Gowda; Anurag Arnab; Jonathan Huang; | eccv | 2024-09-30 |
| 100 | Spiking Transformer with Spatial-Temporal Attention Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we introduce Spiking Transformer with Spatial-Temporal Attention (STAtten), a simple and straightforward architecture designed to integrate spatial and temporal information in self-attention with negligible additional computational load. |
Donghyun Lee; Yuhang Li; Youngeun Kim; Shiting Xiao; Priyadarshini Panda; | arxiv-cs.NE | 2024-09-29 |
| 101 | Multimodal Misinformation Detection By Learning from Synthetic Data with Multimodal LLMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, the generalizability of detectors trained on synthetic data to real-world scenarios remains unclear due to the distribution gap. To address this, we propose learning from synthetic data for detecting real-world multimodal misinformation through two model-agnostic data selection methods that match synthetic and real-world data distributions. |
Fengzhu Zeng; Wenqian Li; Wei Gao; Yan Pang; | arxiv-cs.CL | 2024-09-29 |
| 102 | MetaMath: Integrating Natural Language and Code for Enhanced Mathematical Reasoning in Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we explore fundamental questions related to solving mathematical reasoning problems using natural language and code with state-of-the-art LLMs, including GPT-4o-mini and LLama-3.1-8b-Turbo. |
Xuyuan Xiong; Simeng Han; Ziyue Zhou; Arman Cohan; | arxiv-cs.CL | 2024-09-28 |
| 103 | 3D-CT-GPT: Generating 3D Radiology Reports Through Integration of Large Vision-Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper introduces 3D-CT-GPT, a Visual Question Answering (VQA)-based medical visual language model specifically designed for generating radiology reports from 3D CT scans, particularly chest CTs. |
HAO CHEN et. al. | arxiv-cs.CV | 2024-09-28 |
| 104 | Efficient Federated Intrusion Detection in 5G Ecosystem Using Optimized BERT-based Model Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper proposes a robust intrusion detection system (IDS) using federated learning and large language models (LLMs). |
Frederic Adjewa; Moez Esseghir; Leila Merghem-Boulahia; | arxiv-cs.CR | 2024-09-28 |
| 105 | INSIGHTBUDDY-AI: Medication Extraction and Entity Linking Using Large Language Models and Ensemble Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we investigate state-of-the-art LLMs in text mining tasks on medications and their related attributes such as dosage, route, strength, and adverse effects. |
Pablo Romero; Lifeng Han; Goran Nenadic; | arxiv-cs.CL | 2024-09-28 |
| 106 | Suicide Phenotyping from Clinical Notes in Safety-Net Psychiatric Hospital Using Multi-Label Classification with Pre-Trained Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Pre-trained language models offer promise for identifying suicidality from unstructured clinical narratives. |
ZEHAN LI et. al. | arxiv-cs.CL | 2024-09-27 |
| 107 | Cottention: Linear Transformers With Cosine Attention Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce Cottention, a novel attention mechanism that replaces the softmax operation with cosine similarity. |
Gabriel Mongaras; Trevor Dohm; Eric C. Larson; | arxiv-cs.LG | 2024-09-27 |
| 108 | FoodMLLM-JP: Leveraging Multimodal Large Language Models for Japanese Recipe Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Research on food image understanding using recipe data has been a long-standing focus due to the diversity and complexity of the data. |
Yuki Imajuku; Yoko Yamakata; Kiyoharu Aizawa; | arxiv-cs.CV | 2024-09-27 |
| 109 | Experimental Evaluation of Machine Learning Models for Goal-oriented Customer Service Chatbot with Pipeline Architecture Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we present a tailored experimental evaluation approach for goal-oriented customer service chatbots with pipeline architecture, focusing on three key components: Natural Language Understanding (NLU), dialogue management (DM), and Natural Language Generation (NLG). |
Nurul Ain Nabilah Mohd Isa; Siti Nuraishah Agos Jawaddi; Azlan Ismail; | arxiv-cs.AI | 2024-09-27 |
| 110 | Retrospective Comparative Analysis of Prostate Cancer In-Basket Messages: Responses from Closed-Domain LLM Vs. Clinical Teams Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, responding to these patients’ inquiries has become a significant burden on healthcare workflows, consuming considerable time for clinical care teams. To address this, we introduce RadOnc-GPT, a specialized Large Language Model (LLM) powered by GPT-4 that has been designed with a focus on radiotherapeutic treatment of prostate cancer with advanced prompt engineering, and specifically designed to assist in generating responses. |
YUEXING HAO et. al. | arxiv-cs.AI | 2024-09-26 |
| 111 | Predicting Anchored Text from Translation Memories for Machine Translation Using Deep Learning Methods Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this article, we show that for a large part of those words which are anchored, we can use other techniques that are based on machine learning approaches such as Word2Vec. |
Richard Yue; John E. Ortega; | arxiv-cs.CL | 2024-09-26 |
| 112 | The Application of GPT-4 in Grading Design University Students’ Assignment and Providing Feedback: An Exploratory Study Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study aims to investigate whether GPT-4 can effectively grade assignments for design university students and provide useful feedback. |
Qian Huang; Thijs Willems; King Wang Poon; | arxiv-cs.AI | 2024-09-26 |
| 113 | General Compression Framework for Efficient Transformer Object Tracking Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Thus, we propose a general model compression framework for efficient transformer object tracking, named CompressTracker, to reduce the size of a pre-trained tracking model into a lightweight tracker with minimal performance degradation. |
LINGYI HONG et. al. | arxiv-cs.CV | 2024-09-26 |
| 114 | MASSFormer: Mobility-Aware Spectrum Sensing Using Transformer-Driven Tiered Structure Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we develop a novel mobility-aware transformer-driven tiered structure (MASSFormer) based cooperative spectrum sensing method that effectively models the spatio-temporal dynamics of user movements. |
Dimpal Janu; Sandeep Mandia; Kuldeep Singh; Sandeep Kumar; | arxiv-cs.IT | 2024-09-26 |
| 115 | Beyond Turing Test: Can GPT-4 Sway Experts’ Decisions? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In the post-Turing era, evaluating large language models (LLMs) involves assessing generated text based on readers’ reactions rather than merely its indistinguishability from human-produced content. |
Takehiro Takayanagi; Hiroya Takamura; Kiyoshi Izumi; Chung-Chi Chen; | arxiv-cs.CE | 2024-09-25 |
| 116 | Assessing The Level of Toxicity Against Distinct Groups in Bangla Social Media Comments: A Comprehensive Investigation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study focuses on identifying toxic comments in the Bengali language targeting three specific groups: transgender people, indigenous people, and migrant people, from multiple social media sources. |
Mukaffi Bin Moin; Pronay Debnath; Usafa Akther Rifa; Rijeet Bin Anis; | arxiv-cs.CL | 2024-09-25 |
| 117 | Reducing and Exploiting Data Augmentation Noise Through Meta Reweighting Contrastive Learning for Text Classification Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To boost deep learning models’ performance given augmented data/samples in text classification tasks, we propose a novel framework, which leverages both meta learning and contrastive learning techniques as parts of our design for reweighting the augmented samples and refining their feature representations based on their quality. |
Guanyi Mou; Yichuan Li; Kyumin Lee; | arxiv-cs.CL | 2024-09-25 |
| 118 | Comparing Unidirectional, Bidirectional, and Word2vec Models for Discovering Vulnerabilities in Compiled Lifted Code Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Using a dataset of LLVM functions, we trained a GPT-2 model to generate embeddings, which were subsequently used to build LSTM neural networks to differentiate between vulnerable and non-vulnerable code. |
Gary A. McCully; John D. Hastings; Shengjie Xu; Adam Fortier; | arxiv-cs.CR | 2024-09-25 |
| 119 | GPT-4 As A Homework Tutor Can Improve Student Engagement and Learning Outcomes Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This work contributes to the scarce empirical literature on LLM-based interactive homework in real-world educational settings and offers a practical, scalable solution for improving homework in schools. |
Alessandro Vanzo; Sankalan Pal Chowdhury; Mrinmaya Sachan; | arxiv-cs.CY | 2024-09-24 |
| 120 | Unveiling Language Competence Neurons: A Psycholinguistic Approach to Model Interpretability Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: As large language models (LLMs) become advance in their linguistic capacity, understanding how they capture aspects of language competence remains a significant challenge. This study therefore employs psycholinguistic paradigms, which are well-suited for probing deeper cognitive aspects of language processing, to explore neuron-level representations in language model across three tasks: sound-shape association, sound-gender association, and implicit causality. |
Xufeng Duan; Xinyu Zhou; Bei Xiao; Zhenguang G. Cai; | arxiv-cs.CL | 2024-09-24 |
| 121 | MonoFormer: One Transformer for Both Diffusion and Autoregression Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose to study a simple idea: share one transformer for both autoregression and diffusion. |
CHUYANG ZHAO et. al. | arxiv-cs.CV | 2024-09-24 |
| 122 | SynChart: Synthesizing Charts from Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We construct a large-scale chart dataset, SynChart, which contains approximately 4 million diverse chart images with over 75 million dense annotations, including data tables, code, descriptions, and question-answer sets. We trained a 4.2B chart-expert model using this dataset and achieve near-GPT-4O performance on the ChartQA task, surpassing GPT-4V. |
MENGCHEN LIU et. al. | arxiv-cs.AI | 2024-09-24 |
| 123 | SDBA: A Stealthy and Long-Lasting Durable Backdoor Attack in Federated Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper introduces SDBA, a novel backdoor attack mechanism designed for NLP tasks in FL environments. |
Minyeong Choe; Cheolhee Park; Changho Seo; Hyunil Kim; | arxiv-cs.LG | 2024-09-23 |
| 124 | SOFI: Multi-Scale Deformable Transformer for Camera Calibration with Enhanced Line Queries Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we introduce \textit{multi-Scale defOrmable transFormer for camera calibratIon with enhanced line queries}, SOFI. |
Sebastian Janampa; Marios Pattichis; | arxiv-cs.CV | 2024-09-23 |
| 125 | Towards A Realistic Long-Term Benchmark for Open-Web Research Agents Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present initial results of a forthcoming benchmark for evaluating LLM agents on white-collar tasks of economic value. |
Peter Mühlbacher; Nikos I. Bosse; Lawrence Phillips; | arxiv-cs.CL | 2024-09-23 |
| 126 | Evaluating The Quality of Code Comments Generated By Large Language Models for Novice Programmers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Large Language Models (LLMs) show promise in generating code comments for novice programmers, but their educational effectiveness remains under-evaluated. |
Aysa Xuemo Fan; Arun Balajiee Lekshmi Narayanan; Mohammad Hassany; Jiaze Ke; | arxiv-cs.SE | 2024-09-22 |
| 127 | Normalized Narrow Jump To Conclusions: Normalized Narrow Shortcuts for Parameter Efficient Early Exit Transformer Prediction Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we propose Narrow Jump to Conclusions (NJTC) and Normalized Narrow Jump to Conclusions (N-NJTC) – parameter efficient alternatives to standard linear shortcutting that reduces shortcut parameter count by over 97%. |
Amrit Diggavi Seshadri; | arxiv-cs.AI | 2024-09-21 |
| 128 | Can LLMs Replace Neil DeGrasse Tyson? Evaluating The Reliability of LLMs As Science Communicators Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we focus on evaluating the reliability of current LLMs as science communicators. |
Prasoon Bajpai; Niladri Chatterjee; Subhabrata Dutta; Tanmoy Chakraborty; | arxiv-cs.CL | 2024-09-21 |
| 129 | AI Assistants for Spaceflight Procedures: Combining Generative Pre-Trained Transformer and Retrieval-Augmented Generation on Knowledge Graphs With Augmented Reality Cues Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper describes the capabilities and potential of the intelligent personal assistant (IPA) CORE (Checklist Organizer for Research and Exploration), designed to support astronauts during procedures onboard the International Space Station (ISS), the Lunar Gateway station, and beyond. |
OLIVER BENSCH et. al. | arxiv-cs.AI | 2024-09-21 |
| 130 | The Use of GPT-4o and Other Large Language Models for The Improvement and Design of Self-Assessment Scales for Measurement of Interpersonal Communication Skills Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: OpenAI’s ChatGPT (GPT-4 and GPT-4o) and other Large Language Models (LLMs) like Microsoft’s Copilot, Google’s Gemini 1.5 Pro, and Antrophic’s Claude 3.5 Sonnet can be effectively used in various phases of scientific research. |
Goran Bubaš; | arxiv-cs.AI | 2024-09-21 |
| 131 | On Importance of Pruning and Distillation for Efficient Low Resource NLP Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, we explore the case of the low-resource Indic language Marathi. |
AISHWARYA MIRASHI et. al. | arxiv-cs.CL | 2024-09-21 |
| 132 | Loop-Residual Neural Networks for Iterative Refinement Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we introduce a novel Loop-Residual Neural Network, which achieves better performance by utilizing longer computational time without increasing the model size. |
Kei-Sing Ng; Qingchen Wang; | arxiv-cs.AI | 2024-09-21 |
| 133 | QMOS: Enhancing LLMs for Telecommunication with Question Masked Loss and Option Shuffling Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper introduces QMOS, an innovative approach which uses a Question-Masked loss and Option Shuffling trick to enhance the performance of LLMs in answering Multiple-Choice Questions in the telecommunications domain. |
Blessed Guda; Gabrial Zencha A.; Lawrence Francis; Carlee Joe-Wong; | arxiv-cs.CL | 2024-09-21 |
| 134 | T2M-X: Learning Expressive Text-to-Motion Generation from Partially Annotated Data Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we propose T2M-X, a two-stage method that learns expressive text-to-motion generation from partially annotated data. |
Mingdian Liu; Yilin Liu; Gurunandan Krishnan; Karl S Bayer; Bing Zhou; | arxiv-cs.CV | 2024-09-20 |
| 135 | Prompting Large Language Models for Supporting The Differential Diagnosis of Anemia Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Inspired by clinical guidelines, our study aimed to develop pathways similar to those that can be obtained in clinical guidelines. |
Elisa Castagnari; Lillian Muyama; Adrien Coulet; | arxiv-cs.CL | 2024-09-20 |
| 136 | Applying Pre-trained Multilingual BERT in Embeddings for Improved Malicious Prompt Injection Attacks Detection Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Large language models (LLMs) are renowned for their exceptional capabilities, and applying to a wide range of applications. |
Md Abdur Rahman; Hossain Shahriar; Fan Wu; Alfredo Cuzzocrea; | arxiv-cs.CL | 2024-09-20 |
| 137 | HUT: A More Computation Efficient Fine-Tuning Method With Hadamard Updated Transformation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This approach ensures that the correlation between the original and updated parameters is preserved, leveraging the semantic features learned during pre-training. Building on this paradigm, we present the Hadamard Updated Transformation (HUT) method. |
Geyuan Zhang; Xiaofei Zhou; Chuheng Chen; | arxiv-cs.CL | 2024-09-20 |
| 138 | ‘Since Lawyers Are Males..’: Examining Implicit Gender Bias in Hindi Language Generation By LLMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Large Language Models (LLMs) are increasingly being used to generate text across various languages, for tasks such as translation, customer support, and education. |
Ishika Joshi; Ishita Gupta; Adrita Dey; Tapan Parikh; | arxiv-cs.CL | 2024-09-20 |
| 139 | Drift to Remember Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We hypothesize that representational drift can alleviate catastrophic forgetting in AI during new task acquisition. To test this, we introduce DriftNet, a network designed to constantly explore various local minima in the loss landscape while dynamically retrieving relevant tasks. |
JIN DU et. al. | arxiv-cs.AI | 2024-09-20 |
| 140 | TACO-RL: Task Aware Prompt Compression Optimization with Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, existing prompt compression techniques either rely on sub-optimal metrics such as information entropy or model it as a task-agnostic token classification problem that fails to capture task-specific information. To address these issues, we propose a novel and efficient reinforcement learning (RL) based task-aware prompt compression method. |
SHIVAM SHANDILYA et. al. | arxiv-cs.CL | 2024-09-19 |
| 141 | 3DTopia-XL: Scaling High-quality 3D Asset Generation Via Primitive Diffusion Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Despite recent advancements in 3D generative models, existing methods still face challenges with optimization speed, geometric fidelity, and the lack of assets for physically based rendering (PBR). In this paper, we introduce 3DTopia-XL, a scalable native 3D generative model designed to overcome these limitations. |
ZHAOXI CHEN et. al. | arxiv-cs.CV | 2024-09-19 |
| 142 | $\text{M}^\text{6}(\text{GPT})^\text{3}$: Generating Multitrack Modifiable Multi-Minute MIDI Music from Text Using Genetic Algorithms, Probabilistic Methods and GPT Models in Any Progression and Time Signature Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a genetic algorithm for the generation of melodic elements. |
Jakub Poćwiardowski; Mateusz Modrzejewski; Marek S. Tatara; | arxiv-cs.SD | 2024-09-19 |
| 143 | Introducing The Large Medical Model: State of The Art Healthcare Cost and Risk Prediction with Transformers Trained on Patient Event Sequences Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper introduces the Large Medical Model (LMM), a generative pre-trained transformer (GPT) designed to guide and predict the broad facets of patient care and healthcare administration. |
RICKY SAHU et. al. | arxiv-cs.LG | 2024-09-19 |
| 144 | Mastering Chess with A Transformer Model Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we explore the application of transformer models to chess, focusing on the critical role of the position encoding within the attention mechanism. |
Daniel Monroe; The Leela Chess Zero Team; | arxiv-cs.LG | 2024-09-18 |
| 145 | Recommendation with Generative Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce a taxonomy that categorizes DGMs into three types: ID-driven models, large language models (LLMs), and multimodal models. |
YASHAR DELDJOO et. al. | arxiv-cs.IR | 2024-09-18 |
| 146 | Self-Supervised Pre-training Tasks for An FMRI Time-series Transformer in Autism Detection Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To address over-fitting in small datasets and enhance the model performance, we propose self-supervised pre-training tasks to reconstruct the randomly masked fMRI time-series data, investigating the effects of various masking strategies. |
Yinchi Zhou; Peiyu Duan; Yuexi Du; Nicha C. Dvornek; | arxiv-cs.CV | 2024-09-18 |
| 147 | AlignBot: Aligning VLM-powered Customized Task Planning with User Reminders Through Fine-Tuning for Household Robots Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents AlignBot, a novel framework designed to optimize VLM-powered customized task planning for household robots by effectively aligning with user reminders. |
PENGAN CHEN et. al. | arxiv-cs.RO | 2024-09-18 |
| 148 | Program Slicing in The Era of Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, we investigate the application of large language models (LLMs) to both static and dynamic program slicing, with a focus on Java programs. |
Kimya Khakzad Shahandashti; Mohammad Mahdi Mohajer; Alvine Boaye Belle; Song Wang; Hadi Hemmati; | arxiv-cs.SE | 2024-09-18 |
| 149 | Small Language Models Can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we evaluate the creative fiction writing abilities of a fine-tuned small language model (SLM), BART Large, and compare its performance to humans and two large language models (LLMs): GPT-3.5 and GPT-4o. |
Guillermo Marco; Luz Rello; Julio Gonzalo; | arxiv-cs.CL | 2024-09-17 |
| 150 | American Sign Language to Text Translation Using Transformer and Seq2Seq with LSTM Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study compares the Transformer with the Sequence-to-Sequence (Seq2Seq) model in translating sign language to text. |
Gregorius Guntur Sunardi Putra; Adifa Widyadhani Chanda D’Layla; Dimas Wahono; Riyanarto Sarno; Agus Tri Haryono; | arxiv-cs.CL | 2024-09-17 |
| 151 | Adaptive Large Language Models By Layerwise Attention Shortcuts Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, they are based on simply stacking the same blocks in dozens of layers and processing information sequentially from one block to another. In this paper, we propose to challenge this and introduce adaptive computations for LLM-like setups, which allow the final layer to attend to all of the intermediate layers as it deems fit through the attention mechanism, thereby introducing computational \textbf{attention shortcuts}. |
Prateek Verma; Mert Pilanci; | arxiv-cs.CL | 2024-09-16 |
| 152 | Code Vulnerability Detection: A Comparative Analysis of Emerging Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Our study aims to shed light on the capabilities of LLMs in vulnerability detection, contributing to the enhancement of software security practices across diverse open-source repositories. |
Shaznin Sultana; Sadia Afreen; Nasir U. Eisty; | arxiv-cs.SE | 2024-09-16 |
| 153 | Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, inspired by the recent public release of the GPT-o1 models, we conduct the first study to compare the effectiveness of different versions of the GPT-family models in APR. |
Haichuan Hu; Ye Shang; Guolin Xu; Congqing He; Quanjun Zhang; | arxiv-cs.SE | 2024-09-16 |
| 154 | LLMs for Clinical Risk Prediction Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study compares the efficacy of GPT-4 and clinalytix Medical AI in predicting the clinical risk of delirium development. |
Mohamed Rezk; Patricia Cabanillas Silva; Fried-Michael Dahlweid; | arxiv-cs.CL | 2024-09-16 |
| 155 | SelECT-SQL: Self-correcting Ensemble Chain-of-Thought for Text-to-SQL Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce SelECT-SQL, a novel in-context learning solution that uses an algorithmic combination of chain-of-thought (CoT) prompting, self-correction, and ensemble methods to yield a new state-of-the-art result on challenging Text-to-SQL benchmarks. |
Ke Shen; Mayank Kejriwal; | arxiv-cs.CL | 2024-09-16 |
| 156 | Investigating The Impact of Code Comment Inconsistency on Bug Introducing Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Our research investigates the impact of code-comment inconsistency on bug introduction using large language models, specifically GPT-3.5. |
Shiva Radmanesh; Aaron Imani; Iftekhar Ahmed; Mohammad Moshirpour; | arxiv-cs.SE | 2024-09-16 |
| 157 | CAT: Customized Transformer Accelerator Framework on Versal ACAP Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: It is far more flexible than GPU in hardware customization, and has better and smaller design solution space than traditional FPGA. Therefore, this paper proposes the Customized Transformer Accelerator Framework(CAT), through the CAT framework, a customized Transformer accelerator family can be derived on Versal ACAP, CAT framework has an abstract accelerator architecture design idea, which deconstructs and efficiently maps the Transformer into the hardware, which contains a variety of customizable properties. |
Wenbo Zhang; Yiqi Liu; Zhenshan Bao; | arxiv-cs.AR | 2024-09-15 |
| 158 | GP-GPT: Large Language Model for Gene-Phenotype Mapping Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, the complex traits and heterogeneity of multi-sources genomics data pose significant challenges when adapting these models to the bioinformatics and biomedical field. To address these challenges, we present GP-GPT, the first specialized large language model for genetic-phenotype knowledge representation and genomics relation analysis. |
YANJUN LYU et. al. | arxiv-cs.CL | 2024-09-15 |
| 159 | Leveraging Open-Source Large Language Models for Native Language Identification Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Native Language Identification (NLI) – the task of identifying the native language (L1) of a person based on their writing in the second language (L2) – has applications in forensics, marketing, and second language acquisition. Historically, conventional machine learning approaches that heavily rely on extensive feature engineering have outperformed transformer-based language models on this task. |
Yee Man Ng; Ilia Markov; | arxiv-cs.CL | 2024-09-15 |
| 160 | Evaluating Authenticity and Quality of Image Captions Via Sentiment and Semantic Analyses Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study proposes an evaluation method focused on sentiment and semantic richness. |
Aleksei Krotov; Alison Tebo; Dylan K. Picart; Aaron Dean Algave; | arxiv-cs.CV | 2024-09-14 |
| 161 | Undergrads Are All You Have Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The paper also demonstrates that GPT-UGRD is cheaper and easier to train and operate than transformer models. In this paper, we outline the implementation, application, multi-tenanting, and social implications of using this new model in research and other contexts. |
Ashe Neth; | arxiv-cs.CY | 2024-09-13 |
| 162 | Guiding Vision-Language Model Selection for Visual Question-Answering Across Tasks, Domains, and Knowledge Types Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper introduces a comprehensive framework for evaluating VLMs tailored to VQA tasks in practical settings. |
Neelabh Sinha; Vinija Jain; Aman Chadha; | arxiv-cs.CV | 2024-09-13 |
| 163 | Autoregressive + Chain of Thought = Recurrent: Recurrence’s Role in Language Models’ Computability and A Revisit of Recurrent Transformer Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we thoroughly investigate the influence of recurrent structures in neural models on their reasoning abilities and computability, contrasting the role autoregression plays in the neural models’ computational power. |
Xiang Zhang; Muhammad Abdul-Mageed; Laks V. S. Lakshmanan; | arxiv-cs.CL | 2024-09-13 |
| 164 | Enhanced Online Grooming Detection Employing Context Determination and Message-Level Analysis Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper’s contributions include improved detection methodologies and the potential for application in various scenarios, addressing gaps in current literature and practices. |
Jake Street; Isibor Ihianle; Funminiyi Olajide; Ahmad Lotfi; | arxiv-cs.LG | 2024-09-12 |
| 165 | SDformer: Efficient End-to-End Transformer for Depth Completion Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose a different window-based Transformer architecture for depth completion tasks named Sparse-to-Dense Transformer (SDformer). |
JIAN QIAN et. al. | arxiv-cs.CV | 2024-09-12 |
| 166 | Towards Fairer Health Recommendations: Finding Informative Unbiased Samples Via Word Sense Disambiguation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, some of these terms, especially those related to race and ethnicity, can carry different meanings (e.g., white matter of spinal cord). To address this issue, we propose the use of Word Sense Disambiguation models to refine dataset quality by removing irrelevant sentences. |
GAVIN BUTTS et. al. | arxiv-cs.CL | 2024-09-11 |
| 167 | A Novel Mathematical Framework for Objective Characterization of Ideas Through Vector Embeddings in LLM Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This method suffers from limitations such as human judgment errors, bias, and oversight. Addressing this gap, our study introduces a comprehensive mathematical framework for automated analysis to objectively evaluate the plethora of ideas generated by CAI systems and/or humans. |
B. Sankar; Dibakar Sen; | arxiv-cs.AI | 2024-09-11 |
| 168 | A Fine-grained Sentiment Analysis of App Reviews Using Large Language Models: An Evaluation Study Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Analyzing user reviews for sentiment towards app features can provide valuable insights into users’ perceptions of app functionality and their evolving needs. |
Faiz Ali Shah; Ahmed Sabir; Rajesh Sharma; | arxiv-cs.CL | 2024-09-11 |
| 169 | GroUSE: A Benchmark to Evaluate Evaluators in Grounded Question Answering Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we address the challenges of using LLM-as-a-Judge when evaluating grounded answers generated by RAG systems. |
Sacha Muller; António Loison; Bilel Omrani; Gautier Viaud; | arxiv-cs.CL | 2024-09-10 |
| 170 | FairHome: A Fair Housing and Fair Lending Dataset Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present a Fair Housing and Fair Lending dataset (FairHome): A dataset with around 75,000 examples across 9 protected categories. |
Anusha Bagalkotkar; Aveek Karmakar; Gabriel Arnson; Ondrej Linda; | arxiv-cs.LG | 2024-09-09 |
| 171 | Harmonic Reasoning in Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Large Language Models (LLMs) are becoming very popular and are used for many different purposes, including creative tasks in the arts. |
Anna Kruspe; | arxiv-cs.CL | 2024-09-09 |
| 172 | Retrofitting Temporal Graph Neural Networks with Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose TF-TGN, which uses Transformer decoder as the backbone model for TGNN to enjoy Transformer’s codebase for efficient training. |
QIANG HUANG et. al. | arxiv-cs.LG | 2024-09-09 |
| 173 | NOVI : Chatbot System for University Novice with BERT and LLMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To mitigate the difficulties of university freshmen in adapting to university life, we developed NOVI, a chatbot system based on GPT-4o. |
Yoonji Nam; TaeWoong Seo; Gyeongcheol Shin; Sangji Lee; JaeEun Im; | arxiv-cs.CL | 2024-09-09 |
| 174 | Identifying The Sources of Ideological Bias in GPT Models Through Linguistic Variation in Output Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this article, we provide an original approach to identifying ideological bias in generative models, showing that bias can stem from both the training data and the filtering algorithm. |
Christina Walker; Joan C. Timoneda; | arxiv-cs.CL | 2024-09-09 |
| 175 | Can Large Language Models Unlock Novel Scientific Research Ideas? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study explores the capability of LLMs in generating novel research ideas based on information from research papers. |
Sandeep Kumar; Tirthankar Ghosal; Vinayak Goyal; Asif Ekbal; | arxiv-cs.CL | 2024-09-09 |
| 176 | Low Latency Transformer Inference on FPGAs for Physics Applications with Hls4ml Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study presents an efficient implementation of transformer architectures in Field-Programmable Gate Arrays(FPGAs) using hls4ml. |
ZHIXING JIANG et. al. | arxiv-cs.LG | 2024-09-08 |
| 177 | The Emergence of Large Language Models (LLM) As A Tool in Literature Reviews: An LLM Automated Systematic Review Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Objective: This study aims to summarize the usage of Large Language Models (LLMs) in the process of creating a scientific review. |
Dmitry Scherbakov; Nina Hubig; Vinita Jansari; Alexander Bakumenko; Leslie A. Lenert; | arxiv-cs.DL | 2024-09-06 |
| 178 | Hermes: Memory-Efficient Pipeline Inference for Large Models on Edge Devices Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Based on PIPELOAD mechanism, we present Hermes, a framework optimized for large model inference on edge devices. |
XUEYUAN HAN et. al. | arxiv-cs.DC | 2024-09-06 |
| 179 | LLM-based Multi-agent Poetry Generation in Non-cooperative Environments Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Under the rationale that the learning process of the poetry generation systems should be more human-like and their output more diverse and novel, we introduce a framework based on social learning where we emphasize non-cooperative interactions besides cooperative interactions to encourage diversity. |
Ran Zhang; Steffen Eger; | arxiv-cs.CL | 2024-09-05 |
| 180 | CA-BERT: Leveraging Context Awareness for Enhanced Multi-Turn Chat Interaction Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Traditional models often struggle with determining when additional context is necessary for generating appropriate responses. This paper introduces Context-Aware BERT (CA-BERT), a transformer-based model specifically fine-tuned to address this challenge. |
Minghao Liu; Mingxiu Sui; Yi Nan; Cangqing Wang; Zhijie Zhou; | arxiv-cs.CL | 2024-09-05 |
| 181 | CACER: Clinical Concept Annotations for Cancer Events and Relations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present Clinical Concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48,000 medical problems and drug events and 10,000 drug-problem and problem-problem relations. |
YUJUAN FU et. al. | arxiv-cs.CL | 2024-09-05 |
| 182 | Detecting Calls to Action in Multimodal Content: Analysis of The 2021 German Federal Election Campaign on Instagram Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study investigates the automated classification of Calls to Action (CTAs) within the 2021 German Instagram election campaign to advance the understanding of mobilization in social media contexts. |
Michael Achmann-Denkler; Jakob Fehle; Mario Haim; Christian Wolff; | arxiv-cs.SI | 2024-09-04 |
| 183 | OpenFact at CheckThat! 2024: Combining Multiple Attack Methods for Effective Adversarial Text Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents the experiments and results for the CheckThat! |
WŁODZIMIERZ LEWONIEWSKI et. al. | arxiv-cs.CL | 2024-09-04 |
| 184 | MobileUNETR: A Lightweight End-To-End Hybrid Vision Transformer For Efficient Medical Image Segmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Finally many Transformer based approaches rely primarily on CNN based decoders overlooking the benefits of Transformer based decoding models. Recognizing these limitations, we address the need efficient lightweight solutions by introducing MobileUNETR, which aims to overcome the performance constraints associated with both CNNs and Transformers while minimizing model size, presenting a promising stride towards efficient image segmentation. |
Shehan Perera; Yunus Erzurumlu; Deepak Gulati; Alper Yilmaz; | arxiv-cs.CV | 2024-09-04 |
| 185 | Dialogue You Can Trust: Human and AI Perspectives on Generated Conversations Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Utilizing the GPT-4o API, we generated a diverse dataset of conversations and conducted a two-part experimental analysis. |
Ike Ebubechukwu; Johane Takeuchi; Antonello Ceravola; Frank Joublin; | arxiv-cs.CL | 2024-09-03 |
| 186 | How Privacy-Savvy Are Large Language Models? A Case Study on Compliance and Privacy Technical Review Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper seeks to address this gap by providing a comprehensive case study evaluating LLMs’ performance in privacy-related tasks such as privacy information extraction (PIE), legal and regulatory key point detection (KPD), and question answering (QA) with respect to privacy policies and data protection regulations. We introduce a Privacy Technical Review (PTR) framework, highlighting its role in mitigating privacy risks during the software development life-cycle. |
XICHOU ZHU et. al. | arxiv-cs.CL | 2024-09-03 |
| 187 | Beyond ChatGPT: Enhancing Software Quality Assurance Tasks with Diverse LLMs and Validation Techniques Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: There remains a gap in understanding the performance of various LLMs in this critical domain. This paper aims to address this gap by conducting a comprehensive investigation into the capabilities of several LLMs across two SQA tasks: fault localization and vulnerability detection. |
Ratnadira Widyasari; David Lo; Lizi Liao; | arxiv-cs.SE | 2024-09-02 |
| 188 | The Role of Transformer Models in Advancing Blockchain Technology: A Systematic Survey Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This review aims to provide new perspectives and a research foundation for the integrated development of blockchain technology and machine learning, supporting further innovation and application expansion of blockchain technology. |
TIANXU LIU et. al. | arxiv-cs.LG | 2024-09-02 |
| 189 | Towards Faster Graph Partitioning Via Pre-training and Inductive Inference Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Following the IEEE HPEC Graph Challenge and recent advances in pre-training techniques (e.g., large-language models), we propose PR-GPT (Pre-trained & Refined Graph ParTitioning) based on a novel pre-training & refinement paradigm. |
MENG QIN et. al. | arxiv-cs.LG | 2024-09-01 |
| 190 | Research on LLM Acceleration Using The High-Performance RISC-V Processor Xiangshan (Nanhu Version) Based on The Open-Source Matrix Instruction Set Extension (Vector Dot Product) Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The main contributions of this paper are as follows: For the characteristics of large language models, custom instructions were extended based on the RISC-V instruction set to perform vector dot product calculations, accelerating the computation of large language models on dedicated vector dot product acceleration hardware. |
XU-HAO CHEN et. al. | arxiv-cs.AR | 2024-09-01 |
| 191 | An Empirical Study on Information Extraction Using Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Our results suggest a visible performance gap between GPT-4 and state-of-the-art (SOTA) IE methods. To alleviate this problem, considering the LLMs’ human-like characteristics, we propose and analyze the effects of a series of simple prompt-based methods, which can be generalized to other LLMs and NLP tasks. |
RIDONG HAN et. al. | arxiv-cs.CL | 2024-08-31 |
| 192 | From Text to Emotion: Unveiling The Emotion Annotation Capabilities of LLMs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we explore the potential of Large Language Models (LLMs), specifically GPT4, in automating or assisting emotion annotation. |
Minxue Niu; Mimansa Jaiswal; Emily Mower Provost; | arxiv-cs.CL | 2024-08-30 |
| 193 | Finding Frames with BERT: A Transformer-based Approach to Generic News Frame Detection Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The availability of digital data offers new possibilities for studying how specific aspects of social reality are made more salient in online communication but also raises challenges related to the scaling of framing analysis and its adoption to new research areas (e.g. studying the impact of artificial intelligence-powered systems on representation of societally relevant issues). To address these challenges, we introduce a transformer-based approach for generic news frame detection in Anglophone online content. |
Vihang Jumle; Mykola Makhortykh; Maryna Sydorova; Victoria Vziatysheva; | arxiv-cs.CL | 2024-08-30 |
| 194 | Retrieval-Augmented Natural Language Reasoning for Explainable Visual Question Answering Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we introduce a new VQA-NLE model, ReRe (Retrieval-augmented natural language Reasoning), using leverage retrieval information from the memory to aid in generating accurate answers and persuasive explanations without relying on complex networks and extra datasets. |
Su Hyeon Lim; Minkuk Kim; Hyeon Bae Kim; Seong Tae Kim; | arxiv-cs.CV | 2024-08-30 |
| 195 | Improving Extraction of Clinical Event Contextual Properties from Electronic Health Records: A Comparative Study Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study performs a comparative analysis of various natural language models for medical text classification. |
SHUBHAM AGARWAL et. al. | arxiv-cs.CL | 2024-08-30 |
| 196 | Assessing Generative Language Models in Classification Tasks: Performance and Self-Evaluation Capabilities in The Environmental and Climate Change Domain Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Through this research, we aim to contribute to the ongoing discussion on the utility and effectiveness of generative LMs in addressing some of the planet’s most urgent issues, highlighting their strengths and limitations in the context of ecology and CC. |
Francesca Grasso; Stefano Locci; | arxiv-cs.CL | 2024-08-30 |
| 197 | Can Large Language Models Address Open-Target Stance Detection? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce Open-Target Stance Detection (OTSD), the most realistic task where targets are neither seen during training nor provided as input. |
Abu Ubaida Akash; Ahmed Fahmy; Amine Trabelsi; | arxiv-cs.CL | 2024-08-30 |
| 198 | ProGRes: Prompted Generative Rescoring on ASR N-Best Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper proposes a novel method that uses instruction-tuned LLMs to dynamically expand the n-best speech recognition hypotheses with new hypotheses generated through appropriately-prompted LLMs. |
Ada Defne Tur; Adel Moumen; Mirco Ravanelli; | arxiv-cs.CL | 2024-08-30 |
| 199 | Assessing Large Language Models for Online Extremism Research: Identification, Explanation, and New Knowledge Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study evaluates the performance of Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-Trained Transformers (GPT) in detecting and classifying online domestic extremist posts. We collected social media posts containing far-right and far-left ideological keywords and manually labeled them as extremist or non-extremist. |
Beidi Dong; Jin R. Lee; Ziwei Zhu; Balassubramanian Srinivasan; | arxiv-cs.CL | 2024-08-29 |
| 200 | MAPF-GPT: Imitation Learning for Multi-Agent Pathfinding at Scale Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Following current trends in machine learning, we have created a foundation model for the MAPF problems called MAPF-GPT. |
Anton Andreychuk; Konstantin Yakovlev; Aleksandr Panov; Alexey Skrynnik; | arxiv-cs.MA | 2024-08-29 |
| 201 | Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose Fully Pipelined Distributed Transformer (FPDT) for efficiently training long-context LLMs with extreme hardware efficiency. |
JINGHAN YAO et. al. | arxiv-cs.DC | 2024-08-29 |
| 202 | Can AI Replace Human Subjects? A Large-Scale Replication of Psychological Experiments with LLMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We find that GPT-4 successfully replicates 76.0 percent of main effects and 47.0 percent of interaction effects observed in the original studies, closely mirroring human responses in both direction and significance. |
Ziyan Cui; Ning Li; Huaikang Zhou; | arxiv-cs.CL | 2024-08-29 |
| 203 | FRACTURED-SORRY-Bench: Framework for Revealing Attacks in Conversational Turns Undermining Refusal Efficacy and Defenses Over SORRY-Bench Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper introduces FRACTURED-SORRY-Bench, a framework for evaluating the safety of Large Language Models (LLMs) against multi-turn conversational attacks. |
Aman Priyanshu; Supriti Vijay; | arxiv-cs.CL | 2024-08-28 |
| 204 | Unleashing The Temporal-Spatial Reasoning Capacity of GPT for Training-Free Audio and Language Referenced Video Object Segmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose an Audio-Language-Referenced SAM 2 (AL-Ref-SAM 2) pipeline to explore the training-free paradigm for audio and language-referenced video object segmentation, namely AVS and RVOS tasks. |
SHAOFEI HUANG et. al. | arxiv-cs.CV | 2024-08-28 |
| 205 | The Mamba in The Llama: Distilling and Accelerating Hybrid Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Linear RNN architectures, like Mamba, can be competitive with Transformer models in language modeling while having advantageous deployment characteristics. Given the focus on training large-scale Transformer models, we consider the challenge of converting these pretrained models for deployment. |
Junxiong Wang; Daniele Paliotta; Avner May; Alexander M. Rush; Tri Dao; | arxiv-cs.LG | 2024-08-27 |
| 206 | A Review of Transformer-Based Models for Computer Vision Tasks: Capturing Global Context and Spatial Relationships Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this review paper, we provide an extensive overview of various transformer architectures adapted for computer vision tasks. |
Gracile Astlin Pereira; Muhammad Hussain; | arxiv-cs.CV | 2024-08-27 |
| 207 | Speech Recognition Transformers: Topological-lingualism Perspective Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The paper presents a comprehensive survey of transformer techniques oriented in speech modality. |
Shruti Singh; Muskaan Singh; Virender Kadyan; | arxiv-cs.CL | 2024-08-27 |
| 208 | Evaluating Large Language Models on Spatial Tasks: A Multi-Task Benchmarking Study Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We evaluated multiple models, including OpenAI’s gpt-3.5-turbo, gpt-4o, and ZhipuAI’s glm-4, through a two-phase testing approach. |
LIUCHANG XU et. al. | arxiv-cs.CL | 2024-08-26 |
| 209 | Beyond Detection: Leveraging Large Language Models for Cyber Attack Prediction in IoT Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Despite considerable efforts in attack detection, intrusion detection systems remain mostly reactive, responding to specific patterns or observed anomalies. This work proposes a proactive approach to anticipate and mitigate malicious activities before they cause damage. |
Alaeddine Diaf; Abdelaziz Amara Korba; Nour Elislem Karabadji; Yacine Ghamri-Doudane; | arxiv-cs.CR | 2024-08-26 |
| 210 | One-layer Transformers Fail to Solve The Induction Heads Task Summary Related Papers Related Patents Related Grants Related Venues Related Experts View Abstract: A simple communication complexity argument proves that no one-layer transformer can solve the induction heads task unless its size is exponentially larger than the size sufficient … |
Clayton Sanford; Daniel Hsu; Matus Telgarsky; | arxiv-cs.LG | 2024-08-26 |
| 211 | Vision-Language and Large Language Model Performance in Gastroenterology: GPT, Claude, Llama, Phi, Mistral, Gemma, and Quantized Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Methods: We used 300 gastroenterology board exam-style multiple-choice questions, 138 of which contain images to systematically assess the impact of model configurations and parameters and prompt engineering strategies utilizing GPT-3.5. |
SEYED AMIR AHMAD SAFAVI-NAINI et. al. | arxiv-cs.CL | 2024-08-25 |
| 212 | LowCLIP: Adapting The CLIP Model Architecture for Low-Resource Languages in Multimodal Image Retrieval Task Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To address challenges in vision-language retrieval for low-resource languages, we integrated the CLIP model architecture and employed several techniques to balance computational efficiency with performance. |
Ali Asgarov; Samir Rustamov; | arxiv-cs.CV | 2024-08-25 |
| 213 | Bidirectional Awareness Induction in Autoregressive Seq2Seq Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we introduce the Bidirectional Awareness Induction (BAI), a training method that leverages a subset of elements in the network, the Pivots, to perform bidirectional learning without breaking the autoregressive constraints. |
Jia Cheng Hu; Roberto Cavicchioli; Alessandro Capotondi; | arxiv-cs.CL | 2024-08-25 |
| 214 | Utilizing Large Language Models for Named Entity Recognition in Traditional Chinese Medicine Against COVID-19 Literature: Comparative Study Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Objective: To explore and compare the performance of ChatGPT and other state-of-the-art LLMs on domain-specific NER tasks covering different entity types and domains in TCM against COVID-19 literature. Methods: We established a dataset of 389 articles on TCM against COVID-19, and manually annotated 48 of them with 6 types of entities belonging to 3 domains as the ground truth, against which the NER performance of LLMs can be assessed. |
XU TONG et. al. | arxiv-cs.CL | 2024-08-24 |
| 215 | Preliminary Investigations of A Multi-Faceted Robust and Synergistic Approach in Semiconductor Electron Micrograph Analysis: Integrating Vision Transformers with Large Language and Multimodal Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study introduces an innovative architecture that leverages the generative capabilities of zero-shot prompting in Large Language Models (LLMs) such as GPT-4(language only), the predictive ability of few-shot (in-context) learning in Large Multimodal Models (LMMs) such as GPT-4(V)ision, and fuses knowledge across image based and linguistic insights for accurate nanomaterial category prediction. |
Sakhinana Sagar Srinivas; Geethan Sannidhi; Sreeja Gangasani; Chidaksh Ravuru; Venkataramana Runkana; | arxiv-cs.CV | 2024-08-24 |
| 216 | CNN-Transformer Rectified Collaborative Learning for Medical Image Segmentation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To this end, this paper proposes a CNN-Transformer rectified collaborative learning (CTRCL) framework to learn stronger CNN-based and Transformer-based models for MIS tasks via the bi-directional knowledge transfer between them. |
LANHU WU et. al. | arxiv-cs.CV | 2024-08-24 |
| 217 | Enhancing Multi-hop Reasoning Through Knowledge Erasure in Large Language Model Editing Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Building on the validated hypothesis, we propose a novel knowledge editing method that incorporates a Knowledge Erasure mechanism for Large language model Editing (KELE). |
MENGQI ZHANG et. al. | arxiv-cs.CL | 2024-08-22 |
| 218 | Enhancing Automated Program Repair with Solution Design Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This raises a compelling question: How can we leverage DR scattered across the issue logs to efficiently enhance APR? To investigate this premise, we introduce DRCodePilot, an approach designed to augment GPT-4-Turbo’s APR capabilities by incorporating DR into the prompt instruction. |
JIUANG ZHAO et. al. | arxiv-cs.SE | 2024-08-21 |
| 219 | Mixed Sparsity Training: Achieving 4$\times$ FLOP Reduction for Transformer Pretraining Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Large language models (LLMs) have made significant strides in complex tasks, yet their widespread adoption is impeded by substantial computational demands. |
Pihe Hu; Shaolong Li; Longbo Huang; | arxiv-cs.LG | 2024-08-21 |
| 220 | Hypformer: Exploring Efficient Transformer Fully in Hyperbolic Space Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This stems primarily from: (i) the absence of well-defined modules in hyperbolic space, including linear transformation layers, LayerNorm layers, activation functions, dropout operations, etc. (ii) the quadratic time complexity of the existing hyperbolic self-attention module w.r.t the number of input tokens, which hinders its scalability. To address these challenges, we propose, Hypformer, a novel hyperbolic Transformer based on the Lorentz model of hyperbolic geometry. |
MENGLIN YANG et. al. | kdd | 2024-08-21 |
| 221 | Clinical Context-aware Radiology Report Generation from Medical Images Using Transformers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we investigate the use of the transformer model for radiology report generation from chest X-rays. |
Sonit Singh; | arxiv-cs.CL | 2024-08-21 |
| 222 | BURExtract-Llama: An LLM for Clinical Concept Extraction in Breast Ultrasound Reports Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study presents a pipeline for developing an in-house LLM to extract clinical information from radiology reports. |
YUXUAN CHEN et. al. | arxiv-cs.CL | 2024-08-21 |
| 223 | The Self-Contained Negation Test Set Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this article, we build on Gubelmann and Handschuh (2022), which studies the modification of PLMs’ predictions as a function of the polarity of inputs, in English. |
David Kletz; Pascal Amsili; Marie Candito; | arxiv-cs.CL | 2024-08-21 |
| 224 | Maximum-Entropy Regularized Decision Transformer with Reward Relabelling for Dynamic Recommendation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In response, we introduce a novel methodology named Max-Entropy enhanced Decision Transformer with Reward Relabeling for Offline RLRS (EDT4Rec). |
Xiaocong Chen; Siyu Wang; Lina Yao; | kdd | 2024-08-21 |
| 225 | GSTran: Joint Geometric and Semantic Coherence for Point Cloud Segmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In parallel, inaccurate modeling of long-distance contextual dependencies when utilizing global information can also impact model performance. To address these issues, we propose GSTran, a novel transformer network tailored for the segmentation task. |
ABIAO LI et. al. | arxiv-cs.CV | 2024-08-21 |
| 226 | Selene: Pioneering Automated Proof in Software Verification Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce Selene in this paper, which is the first project-level automated proof benchmark constructed based on the real-world industrial-level operating system microkernel, seL4. |
Lichen Zhang; Shuai Lu; Nan Duan; | acl | 2024-08-20 |
| 227 | Self-Evolving GPT: A Lifelong Autonomous Experiential Learner Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, they rely on manual efforts to acquire and apply such experience for each task, which is not feasible for the growing demand for LLMs and the variety of user questions. To address this issue, we design a lifelong autonomous experiential learning framework based on LLMs to explore whether LLMs can imitate human ability for learning and utilizing experience. |
JINGLONG GAO et. al. | acl | 2024-08-20 |
| 228 | D2LLM: Decomposed and Distilled Large Language Models for Semantic Search Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present D2LLMs�Decomposed and Distilled LLMs for semantic search�that combines the best of both worlds. |
Zihan Liao; Hang Yu; Jianguo Li; Jun Wang; Wei Zhang; | acl | 2024-08-20 |
| 229 | Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Subsequently, methods leveraging pre-trained language models like BERT have been developed, which require less data and yield enhanced performance. |
YUCHENG RUAN et. al. | arxiv-cs.CL | 2024-08-20 |
| 230 | The MERSA Dataset and A Transformer-Based Approach for Speech Emotion Recognition Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study introduces the Multimodal Emotion Recognition and Sentiment Analysis (MERSA) dataset, which includes both natural and scripted speech recordings, transcribed text, physiological data, and self-reported emotional surveys from 150 participants collected over a two-week period. |
Enshi Zhang; Rafael Trujillo; Christian Poellabauer; | acl | 2024-08-20 |
| 231 | Acquiring Clean Language Models from Backdoor Poisoned Datasets By Downscaling Frequency Space Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we investigate the learning mechanisms of backdoor LMs in the frequency space by Fourier analysis. |
Zongru Wu; Zhuosheng Zhang; Pengzhou Cheng; Gongshen Liu; | acl | 2024-08-20 |
| 232 | ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose ChiMed-GPT, a new benchmark LLM designed explicitly for Chinese medical domain, and undergoes a comprehensive training regime with pre-training, SFT, and RLHF. |
Yuanhe Tian; Ruyi Gan; Yan Song; Jiaxing Zhang; Yongdong Zhang; | acl | 2024-08-20 |
| 233 | Dependency Transformer Grammars: Integrating Dependency Structures Into Transformer Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Syntactic Transformer language models aim to achieve better generalization through simultaneously modeling syntax trees and sentences. |
Yida Zhao; Chao Lou; Kewei Tu; | acl | 2024-08-20 |
| 234 | Language Models Can Exploit Cross-Task In-context Learning for Data-Scarce Novel Tasks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper investigates whether LLMs can generalize from labeled examples of predefined tasks to novel tasks. |
Anwoy Chatterjee; Eshaan Tanwar; Subhabrata Dutta; Tanmoy Chakraborty; | acl | 2024-08-20 |
| 235 | GPT Is Not An Annotator: The Necessity of Human Annotation in Fairness Benchmark Construction Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper explores whether an LLM (specifically, GPT-3. |
Virginia Felkner; Jennifer Thompson; Jonathan May; | acl | 2024-08-20 |
| 236 | Automated Detection of Algorithm Debt in Deep Learning Frameworks: An Empirical Study Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Aim: Our goal is to improve AD detection performance of various ML/DL models. |
Emmanuel Iko-Ojo Simon; Chirath Hettiarachchi; Alex Potanin; Hanna Suominen; Fatemeh Fard; | arxiv-cs.SE | 2024-08-20 |
| 237 | Characterizing Similarities and Divergences in Conversational Tones in Humans and LLMs By Sampling with People Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Inspired by methods from cognitive science, we propose an iterative method for simultaneously eliciting conversational tones and sentences, where participants alternate between two tasks: (1) one participant identifies the tone of a given sentence and (2) a different participant generates a sentence based on that tone. |
Dun-Ming Huang; Pol Van Rijn; Ilia Sucholutsky; Raja Marjieh; Nori Jacoby; | acl | 2024-08-20 |
| 238 | CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Incorrect initial angles between Q and K can cause misestimation in modeling rotary position embedding of the closest tokens. To address this issue, we propose Collinear Constrained Attention mechanism, namely CoCA. |
SHIYI ZHU et. al. | acl | 2024-08-20 |
| 239 | CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, the absence of a comprehensive benchmark impedes progress in this field. To bridge this gap, we introduce CharacterEval, a Chinese benchmark for comprehensive RPCA assessment, complemented by a tailored high-quality dataset. |
QUAN TU et. al. | acl | 2024-08-20 |
| 240 | MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we present a novel **map**-guided **GPT**-based agent, dubbed **MapGPT**, which introduces an online linguistic-formed map to encourage the global exploration. |
JIAQI CHEN et. al. | acl | 2024-08-20 |
| 241 | Advancing Parameter Efficiency in Fine-tuning Via Representation Editing IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Despite the promising performance of current PEFT methods, they present challenges in hyperparameter selection, such as determining the rank of LoRA or Adapter, or specifying the length of soft prompts. In addressing these challenges, we propose a novel approach to fine-tuning neural models, termed Representation EDiting (RED), which scales and biases the representation produced at each layer. |
MULING WU et. al. | acl | 2024-08-20 |
| 242 | MultiLegalPile: A 689GB Multilingual Legal Corpus IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, so far, few datasets are available for specialized critical domains such as law and the available ones are often small and only in English. To fill this gap, we curate and release MultiLegalPile, a 689GB corpus in 24 languages from 17 jurisdictions. |
Joel Niklaus; Veton Matoshi; Matthias St�rmer; Ilias Chalkidis; Daniel Ho; | acl | 2024-08-20 |
| 243 | An Empirical Analysis on Large Language Models in Debate Evaluation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, we investigate the capabilities and inherent biases of advanced large language models (LLMs) such as GPT-3. |
Xinyi Liu; Pinxin Liu; Hangfeng He; | acl | 2024-08-20 |
| 244 | Your Transformer Is Secretly Linear Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper reveals a novel linear characteristic exclusive to transformer decoders, including models like GPT, LLaMA, OPT, BLOOM and others. |
ANTON RAZZHIGAEV et. al. | acl | 2024-08-20 |
| 245 | MELA: Multilingual Evaluation of Linguistic Acceptability Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we present the largest benchmark to date on linguistic acceptability: Multilingual Evaluation of Linguistic Acceptability�MELA, with 46K samples covering 10 languages from a diverse set of language families. |
ZIYIN ZHANG et. al. | acl | 2024-08-20 |
| 246 | Mission: Impossible Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Here, we develop a set of synthetic impossible languages of differing complexity, each designed by systematically altering English data with unnatural word orders and grammar rules. |
Julie Kallini; Isabel Papadimitriou; Richard Futrell; Kyle Mahowald; Christopher Potts; | acl | 2024-08-20 |
| 247 | Tree Transformer�s Disambiguation Ability of Prepositional Phrase Attachment and Garden Path Effects Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: For comparison we evaluate a pretrained supervised BiLSTM-based model trained on constituency parsing as sequence labelling (G�mez-Rodr�guez and Vilares, 2018). |
Lingling Zhou; Suzan Verberne; Gijs Wijnholds; | acl | 2024-08-20 |
| 248 | Linear Transformers with Learnable Kernel Functions Are Better In-Context Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Mirroring the Transformer�s in-context adeptness, it became a strong contender in the field. In our work, we present a singular, elegant alteration to the Based kernel that amplifies its In-Context Learning abilities evaluated with the Multi-Query Associative Recall task and overall language modeling process, as demonstrated on the Pile dataset. |
YAROSLAV AKSENOV et. al. | acl | 2024-08-20 |
| 249 | Crafting Tomorrow’s Headlines: Neural News Generation and Detection in English, Turkish, Hungarian, and Persian Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we take a significant step by introducing a benchmark dataset designed for neural news detection in four languages: English, Turkish, Hungarian, and Persian. |
CEM ÜYÜK et. al. | arxiv-cs.CL | 2024-08-20 |
| 250 | Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we present a method that is able to distill a pretrained Transformer architecture into alternative architectures such as state space models (SSMs). |
Aviv Bick; Kevin Y. Li; Eric P. Xing; J. Zico Kolter; Albert Gu; | arxiv-cs.LG | 2024-08-19 |
| 251 | Rhyme-aware Chinese Lyric Generator Based on GPT Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To enhance the rhyming quality of generated lyrics, we incorporate integrated rhyme information into our model, thereby improving lyric generation performance. |
YIXIAO YUAN et. al. | arxiv-cs.CL | 2024-08-19 |
| 252 | GPT-based Textile Pilling Classification Using 3D Point Cloud Data Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Based on PointGPT, the GPT-like big model of point cloud analysis, we incorporate the global features of the input point cloud extracted from the non-parametric network into it, thus proposing the PointGPT+NN model. |
Yu Lu; YuYu Chen; Gang Zhou; Zhenghua Lan; | arxiv-cs.CV | 2024-08-19 |
| 253 | How Well Do Large Language Models Serve As End-to-End Secure Code Producers? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents a systematic investigation into LLMs’ inherent potential to generate code with fewer vulnerabilities. |
JIANIAN GONG et. al. | arxiv-cs.SE | 2024-08-19 |
| 254 | Demystifying The Communication Characteristics for Distributed Transformer Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We use GPT-based language models as a case study of the transformer architecture due to their ubiquity. |
QUENTIN ANTHONY et. al. | arxiv-cs.DC | 2024-08-19 |
| 255 | STransformer: A Modular Approach for Extracting Inter-Sequential and Temporal Information for Time-Series Forecasting Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we review and categorize existing Transformer-based models into two main types: (1) modifications to the model structure and (2) modifications to the input data. |
JIAHENG YIN et. al. | arxiv-cs.LG | 2024-08-19 |
| 256 | Active Learning for Identifying Disaster-Related Tweets: A Comparison with Keyword Filtering and Generic Fine-Tuning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We compare a keyword filtering approach, a RoBERTa model fine-tuned with generic data from CrisisLex, a base RoBERTa model trained with AL and a fine-tuned RoBERTa model trained with AL regarding classification performance. |
David Hanny; Sebastian Schmidt; Bernd Resch; | arxiv-cs.CL | 2024-08-19 |
| 257 | Attention Is A Smoothed Cubic Spline Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We highlight a perhaps important but hitherto unobserved insight: The attention module in a transformer is a smoothed cubic spline. |
Zehua Lai; Lek-Heng Lim; Yucong Liu; | arxiv-cs.AI | 2024-08-18 |
| 258 | A Strategy to Combine 1stGen Transformers and Open LLMs for Automatic Text Classification Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study compares three 1stTRs (BERT, RoBERTa, and BART) with two open LLMs (Llama 2 and Bloom) across 11 sentiment analysis datasets. |
CLAUDIO M. V. DE ANDRADE et. al. | arxiv-cs.CL | 2024-08-18 |
| 259 | A Unified Framework for Interpretable Transformers Using PDEs and Information Theory Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents a novel unified theoretical framework for understanding Transformer architectures by integrating Partial Differential Equations (PDEs), Neural Information Flow Theory, and Information Bottleneck Theory. |
Yukun Zhang; | arxiv-cs.LG | 2024-08-18 |
| 260 | From Specifications to Prompts: On The Future of Generative LLMs in Requirements Engineering Summary Related Papers Related Patents Related Grants Related Venues Related Experts View Abstract: Generative LLMs, such as GPT, have the potential to revolutionize Requirements Engineering (RE) by automating tasks in new ways. This column explores the novelties and introduces … |
Andreas Vogelsang; | arxiv-cs.SE | 2024-08-17 |
| 261 | See What LLMs Cannot Answer: A Self-Challenge Framework for Uncovering LLM Weaknesses Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Designing tasks and finding LLMs’ limitations are becoming increasingly important. In this paper, we investigate the question of whether an LLM can discover its own limitations from the errors it makes. |
YULONG CHEN et. al. | arxiv-cs.CL | 2024-08-16 |
| 262 | The Fellowship of The LLMs: Multi-Agent Workflows for Synthetic Preference Optimization Dataset Generation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper presents synthetic Preference Optimization (PO) datasets generated using multi-agent workflows and evaluates the effectiveness and potential of these workflows in the dataset generation process. |
Samee Arif; Sualeha Farid; Abdul Hameed Azeemi; Awais Athar; Agha Ali Raza; | arxiv-cs.CL | 2024-08-16 |
| 263 | MAT-SED: A Masked Audio Transformer with Masked-Reconstruction Based Pre-training for Sound Event Detection Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose a pure Transformer-based SED model with masked-reconstruction based pre-training, termed MAT-SED. |
Pengfei Cai; Yan Song; Kang Li; Haoyu Song; Ian McLoughlin; | arxiv-cs.SD | 2024-08-16 |
| 264 | Mamba Retriever: Utilizing Mamba for Effective and Efficient Dense Retrieval Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In the information retrieval (IR) area, dense retrieval (DR) models use deep learning techniques to encode queries and passages into embedding space to compute their semantic relations. |
Hanqi Zhang; Chong Chen; Lang Mei; Qi Liu; Jiaxin Mao; | arxiv-cs.IR | 2024-08-15 |
| 265 | Extracting Sentence Embeddings from Pretrained Transformer Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Methods: Given 110M parameters BERT’s hidden representations from multiple layers and multiple tokens we tried various ways to extract optimal sentence representations. |
Lukas Stankevičius; Mantas Lukoševičius; | arxiv-cs.CL | 2024-08-15 |
| 266 | Leveraging Web-Crawled Data for High-Quality Fine-Tuning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We argue that although the web-crawled data often has formatting errors causing semantic inaccuracies, it can still serve as a valuable source for high-quality supervised fine-tuning in specific domains without relying on advanced models like GPT-4. |
Jing Zhou; Chenglin Jiang; Wei Shen; Xiao Zhou; Xiaonan He; | arxiv-cs.CL | 2024-08-15 |
| 267 | Transformers and Large Language Models for Efficient Intrusion Detection Systems: A Comprehensive Survey Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This survey paper provides a comprehensive analysis of the utilization of Transformers and LLMs in cyber-threat detection systems. |
Hamza Kheddar; | arxiv-cs.CR | 2024-08-14 |
| 268 | MultiSurf-GPT: Facilitating Context-Aware Reasoning with Large-Scale Language Models for Multimodal Surface Sensing Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose MultiSurf-GPT, which utilizes the advanced capabilities of GPT-4o to process and interpret diverse modalities (radar, microscope and multispectral data) uniformly based on prompting strategies (zero-shot and few-shot prompting). |
YONGQUAN HU et. al. | arxiv-cs.HC | 2024-08-14 |
| 269 | Evaluating Cultural Adaptability of A Large Language Model Via Simulation of Synthetic Personas Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Our analysis shows that specifying a person’s country of residence improves GPT-3.5’s alignment with their responses. |
Louis Kwok; Michal Bravansky; Lewis D. Griffin; | arxiv-cs.CL | 2024-08-13 |
| 270 | Generative AI for Automatic Topic Labelling Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper proposes to assess the reliability of three LLMs, namely flan, GPT-4o, and GPT-4 mini for topic labelling. |
Diego Kozlowski; Carolina Pradier; Pierre Benz; | arxiv-cs.CL | 2024-08-13 |
| 271 | Pragmatic Inference of Scalar Implicature By LLMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study investigates how Large Language Models (LLMs), particularly BERT (Devlin et al., 2019) and GPT-2 (Radford et al., 2019), engage in pragmatic inference of scalar implicature, such as some. |
Ye-eun Cho; Seong mook Kim; | arxiv-cs.CL | 2024-08-13 |
| 272 | Sumotosima: A Framework and Dataset for Classifying and Summarizing Otoscopic Images Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a novel resource efficient deep learning and transformer based framework, Sumotosima (Summarizer for otoscopic images), an end-to-end pipeline for classification followed by summarization. |
Eram Anwarul Khan; Anas Anwarul Haq Khan; | arxiv-cs.CV | 2024-08-13 |
| 273 | A RAG-Based Question-Answering Solution for Cyber-Attack Investigation and Attribution Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In the constantly evolving field of cybersecurity, it is imperative for analysts to stay abreast of the latest attack trends and pertinent information that aids in the investigation and attribution of cyber-attacks. In this work, we introduce the first question-answering (QA) model and its application that provides information to the cybersecurity experts about cyber-attacks investigations and attribution. |
Sampath Rajapaksha; Ruby Rani; Erisa Karafili; | arxiv-cs.CR | 2024-08-12 |
| 274 | A Perspective on Large Language Models, Intelligent Machines, and Knowledge Acquisition Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, there is a huge gap between LLM’s and human capabilities for understanding abstract concepts and reasoning. We discuss these issues in a larger philosophical context of human knowledge acquisition and the Turing test. |
Vladimir Cherkassky; Eng Hock Lee; | arxiv-cs.CL | 2024-08-12 |
| 275 | Large Language Models for Secure Code Assessment: A Multi-Language Empirical Study Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we evaluate the effectiveness of LLMs in detecting and classifying Common Weakness Enumerations (CWE) using different prompt and role strategies. |
Kohei Dozono; Tiago Espinha Gasiba; Andrea Stocco; | arxiv-cs.SE | 2024-08-12 |
| 276 | The Language of Trauma: Modeling Traumatic Event Descriptions Across Domains with Explainable AI Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, studies traditionally focus on a single aspect of trauma, often neglecting the transferability of findings across different scenarios. We address this gap by training language models with progressing complexity on trauma-related datasets, including genocide-related court data, a Reddit dataset on post-traumatic stress disorder (PTSD), counseling conversations, and Incel forum posts. |
Miriam Schirmer; Tobias Leemann; Gjergji Kasneci; Jürgen Pfeffer; David Jurgens; | arxiv-cs.CL | 2024-08-12 |
| 277 | Spacetime $E(n)$-Transformer: Equivariant Attention for Spatio-temporal Graphs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce an $E(n)$-equivariant Transformer architecture for spatio-temporal graph data. |
Sergio G. Charles; | arxiv-cs.LG | 2024-08-12 |
| 278 | Is It A Work or Leisure Travel? Applying Text Classification to Identify Work-related Travel on Social Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, we propose a model to predict whether a trip is leisure or work-related, utilizing state-of-the-art Automatic Text Classification (ATC) models such as BERT, RoBERTa, and BART to enhance the understanding of user travel purposes and improve recommendation accuracy in specific travel scenarios. |
Lucas Félix; Washington Cunha; Jussara Almeida; | arxiv-cs.SI | 2024-08-12 |
| 279 | Body Transformer: Leveraging Robot Embodiment for Policy Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Despite notable evidence of successful deployment of this architecture in the context of robot learning, we claim that vanilla transformers do not fully exploit the structure of the robot learning problem. Therefore, we propose Body Transformer (BoT), an architecture that leverages the robot embodiment by providing an inductive bias that guides the learning process. |
Carmelo Sferrazza; Dun-Ming Huang; Fangchen Liu; Jongmin Lee; Pieter Abbeel; | arxiv-cs.RO | 2024-08-12 |
| 280 | Cross-Lingual Conversational Speech Summarization with Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We build a baseline cascade-based system using open-source speech recognition and machine translation models. |
Max Nelson; Shannon Wotherspoon; Francis Keith; William Hartmann; Matthew Snover; | arxiv-cs.CL | 2024-08-12 |
| 281 | Chain of Condition: Construct, Verify and Solve Conditions for Conditional Question Answering Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Existing approaches struggle with CQA due to two challenges: (1) precisely identifying necessary conditions and the logical relationship, and (2) verifying conditions to detect any that are missing. In this paper, we propose a novel prompting approach, Chain of condition, by first identifying all conditions and constructing their logical relationships explicitly according to the document, then verifying whether these conditions are satisfied, finally solving the logical expression to indicate any missing conditions and generating the answer accordingly. |
Jiuheng Lin; Yuxuan Lai; Yansong Feng; | arxiv-cs.CL | 2024-08-10 |
| 282 | Evaluating The Capability of Large Language Models to Personalize Science Texts for Diverse Middle-school-age Learners Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, GPT-4 was used to profile student learning preferences based on choices made during a training session. |
Michael Vaccaro Jr; Mikayla Friday; Arash Zaghi; | arxiv-cs.HC | 2024-08-09 |
| 283 | Retrieval-augmented Code Completion for Local Projects Using Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we focus on using LLMs with around 160 million parameters that are suitable for local execution and augmentation with retrieval from local projects. |
Marko Hostnik; Marko Robnik-Šikonja; | arxiv-cs.SE | 2024-08-09 |
| 284 | From Text to Insight: Leveraging Large Language Models for Performance Evaluation in Management Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Through comparative analyses across two studies, including various task performance outputs, we demonstrate that LLMs can serve as a reliable and even superior alternative to human raters in evaluating knowledge-based performance outputs, which are a key contribution of knowledge workers. |
Ning Li; Huaikang Zhou; Mingze Xu; | arxiv-cs.CL | 2024-08-09 |
| 285 | Transformer Explainer: Interactive Learning of Text-Generative Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present Transformer Explainer, an interactive visualization tool designed for non-experts to learn about Transformers through the GPT-2 model. |
AEREE CHO et. al. | arxiv-cs.LG | 2024-08-08 |
| 286 | Enhancing Journalism with AI: A Study of Contextualized Image Captioning for News Articles Using LLMs and LMMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study explores how LLMs and LMMs can assist journalistic practice by generating contextualised captions for images accompanying news articles. |
Aliki Anagnostopoulou; Thiago Gouvea; Daniel Sonntag; | arxiv-cs.CL | 2024-08-08 |
| 287 | Towards Explainable Network Intrusion Detection Using Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Current state-of-the-art NIDS rely on artificial benchmarking datasets, resulting in skewed performance when applied to real-world networking environments. Therefore, we compare the GPT-4 and LLama3 models against traditional architectures and transformer-based models to assess their ability to detect malicious NetFlows without depending on artificially skewed datasets, but solely on their vast pre-trained acquired knowledge. |
Paul R. B. Houssel; Priyanka Singh; Siamak Layeghy; Marius Portmann; | arxiv-cs.CR | 2024-08-08 |
| 288 | Bi-Level Spatial and Channel-aware Transformer for Learned Image Compression Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, these nonlinear approaches frequently overlook the frequency characteristics of images, which limits their compression efficiency. To address this issue, we propose a novel Transformer-based image compression method that enhances the transformation stage by considering frequency components within the feature map. |
Hamidreza Soltani; Erfan Ghasemi; | arxiv-cs.CV | 2024-08-07 |
| 289 | A Comparison of LLM Finetuning Methods & Evaluation Metrics with Travel Chatbot Use Case Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We used two pretrained LLMs utilized for fine-tuning research: LLaMa 2 7B, and Mistral 7B. |
Sonia Meyer; Shreya Singh; Bertha Tam; Christopher Ton; Angel Ren; | arxiv-cs.CL | 2024-08-07 |
| 290 | Evaluating Source Code Quality with Large Languagem Models: A Comparative Study Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper aims to describe and analyze the results obtained using LLMs as a static analysis tool, evaluating the overall quality of code. |
Igor Regis da Silva Simões; Elaine Venson; | arxiv-cs.SE | 2024-08-07 |
| 291 | Image-to-LaTeX Converter for Mathematical Formulas and Text Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this project, we train a vision encoder-decoder model to generate LaTeX code from images of mathematical formulas and text. |
Daniil Gurgurov; Aleksey Morshnev; | arxiv-cs.CL | 2024-08-07 |
| 292 | Accuracy and Consistency of LLMs in The Registered Dietitian Exam: The Impact of Prompt Engineering and Knowledge Retrieval Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose to employ the Registered Dietitian (RD) exam to conduct a standard and comprehensive evaluation of state-of-the-art LLMs, GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro, assessing both accuracy and consistency in nutrition queries. |
Iman Azimi; Mohan Qi; Li Wang; Amir M. Rahmani; Youlin Li; | arxiv-cs.CL | 2024-08-06 |
| 293 | Evaluating The Translation Performance of Large Language Models Based on Euas-20 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Despite the significant progress in translation performance achieved by large language models, machine translation still faces many challenges. Therefore, in this paper, we construct the dataset Euas-20 to evaluate the performance of large language models on translation tasks, the translation ability on different languages, and the effect of pre-training data on the translation ability of LLMs for researchers and developers. |
Yan Huang; Wei Liu; | arxiv-cs.CL | 2024-08-06 |
| 294 | Training LLMs to Recognize Hedges in Spontaneous Narratives Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: After an error analysis on the top performing approaches, we used an LLM-in-the-Loop approach to improve the gold standard coding, as well as to highlight cases in which hedges are ambiguous in linguistically interesting ways that will guide future research. |
Amie J. Paige; Adil Soubki; John Murzaku; Owen Rambow; Susan E. Brennan; | arxiv-cs.CL | 2024-08-06 |
| 295 | HeTraX: Energy Efficient 3D Heterogeneous Manycore Architecture for Transformer Acceleration Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose the design of a three-dimensional heterogeneous architecture referred to as HeTraX specifically optimized to accelerate end-to-end transformer models. |
Pratyush Dhingra; Janardhan Rao Doppa; Partha Pratim Pande; | arxiv-cs.AR | 2024-08-06 |
| 296 | Context Conquers Parameters: Outperforming Proprietary LLM in Commit Message Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, the use of proprietary LLMs like GPT-4 in coding tasks raises privacy and sustainability concerns, which may hinder their industrial adoption. Considering that open-source LLMs have achieved competitive performance in developer tasks such as compiler validation, this study investigates whether they can be used to generate commit messages that are comparable with OMG. |
Aaron Imani; Iftekhar Ahmed; Mohammad Moshirpour; | arxiv-cs.SE | 2024-08-05 |
| 297 | PTM4Tag+: Tag Recommendation of Stack Overflow Posts with Pre-trained Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Inspired by the recent success of pre-trained models (PTMs) in natural language processing (NLP), we present PTM4Tag+, a tag recommendation framework for Stack Overflow posts that utilizes PTMs in language modeling. |
JUNDA HE et. al. | arxiv-cs.SE | 2024-08-05 |
| 298 | Evaluating The Performance of Large Language Models for SDG Mapping (Technical Report) Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, we compare the performance of various language models on the Sustainable Development Goal (SDG) mapping task, using the output of GPT-4o as the baseline. |
Hui Yin; Amir Aryani; Nakul Nambiar; | arxiv-cs.LG | 2024-08-04 |
| 299 | AVESFormer: Efficient Transformer Design for Real-Time Audio-Visual Segmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce AVESFormer, the first real-time Audio-Visual Efficient Segmentation transformer that achieves fast, efficient and light-weight simultaneously. |
ZILI WANG et. al. | arxiv-cs.CV | 2024-08-03 |
| 300 | MiniCPM-V: A GPT-4V Level MLLM on Your Phone Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we present MiniCPM-V, a series of efficient MLLMs deployable on end-side devices. |
YUAN YAO et. al. | arxiv-cs.CV | 2024-08-03 |
| 301 | TCR-GPT: Integrating Autoregressive Model and Reinforcement Learning for T-Cell Receptor Repertoires Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce TCR-GPT, a probabilistic model built on a decoder-only transformer architecture, designed to uncover and replicate sequence patterns in TCR repertoires. |
Yicheng Lin; Dandan Zhang; Yun Liu; | arxiv-cs.LG | 2024-08-02 |
| 302 | Reconsidering Token Embeddings with The Definitions for Pre-trained Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study first analyzes fine-tuning dynamics of a PLM, BART-large, and demonstrates its robustness against degeneration. On the basis of this finding, we propose DefinitionEMB, a method that utilizes definitions to construct isotropically distributed and semantics-related token embeddings for PLMs while maintaining original robustness during fine-tuning. |
Ying Zhang; Dongyuan Li; Manabu Okumura; | arxiv-cs.CL | 2024-08-02 |
| 303 | Toward Automatic Relevance Judgment Using Vision–Language Models for Image–Text Retrieval Evaluation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Vision–Language Models (VLMs) have demonstrated success across diverse applications, yet their potential to assist in relevance judgments remains uncertain. |
Jheng-Hong Yang; Jimmy Lin; | arxiv-cs.IR | 2024-08-02 |
| 304 | High-Throughput Phenotyping of Clinical Text Using Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We conduct a performance comparison of GPT-4 and GPT-3.5-Turbo. |
Daniel B. Hier; S. Ilyas Munzir; Anne Stahlfeld; Tayo Obafemi-Ajayi; Michael D. Carrithers; | arxiv-cs.CL | 2024-08-02 |
| 305 | Advancing Mental Health Pre-Screening: A New Custom GPT for Psychological Distress Assessment Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study introduces ‘Psycho Analyst’, a custom GPT model based on OpenAI’s GPT-4, optimized for pre-screening mental health disorders. |
Jinwen Tang; Yi Shang; | arxiv-cs.CY | 2024-08-02 |
| 306 | MNAT-Net: Multi-Scale Neighborhood Aggregation Transformer Network for Point Cloud Classification and Segmentation Summary Related Papers Related Patents Related Grants Related Venues Related Experts View Abstract: Accurate understanding of 3D objects in complex scenes plays essential roles in the fields of intelligent transportation and autonomous driving technology. Recent deep neural … |
Xuchu Wang; Yue Yuan; | IEEE Transactions on Intelligent Transportation Systems | 2024-08-01 |
| 307 | Leveraging Large Language Models (LLMs) for Traffic Management at Urban Intersections: The Case of Mixed Traffic Scenarios Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study explores the ability of a Large Language Model (LLM), specifically, GPT-4o-mini to improve traffic management at urban intersections. |
Sari Masri; Huthaifa I. Ashqar; Mohammed Elhenawy; | arxiv-cs.CL | 2024-08-01 |
| 308 | Granting GPT-4 License and Opportunity: Enhancing Accuracy and Confidence Estimation for Few-Shot Event Detection Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The present effort explores methods for effective confidence estimation with GPT-4 with few-shot learning for event detection in the BETTER ontology as a vehicle. |
Steven Fincke; Adrien Bibal; Elizabeth Boschee; | arxiv-cs.AI | 2024-08-01 |
| 309 | OmniParser for Pure Vision Based GUI Agent Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, we argue that the power multimodal models like GPT-4V as a general agent on multiple operating systems across different applications is largely underestimated due to the lack of a robust screen parsing technique capable of: 1) reliably identifying interactable icons within the user interface, and 2) understanding the semantics of various elements in a screenshot and accurately associate the intended action with the corresponding region on the screen. To fill these gaps, we introduce \textsc{OmniParser}, a comprehensive method for parsing user interface screenshots into structured elements, which significantly enhances the ability of GPT-4V to generate actions that can be accurately grounded in the corresponding regions of the interface. |
Yadong Lu; Jianwei Yang; Yelong Shen; Ahmed Awadallah; | arxiv-cs.CV | 2024-07-31 |
| 310 | The Llama 3 Herd of Models IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper presents a new set of foundation models, called Llama 3. |
ABHIMANYU DUBEY et. al. | arxiv-cs.AI | 2024-07-31 |
| 311 | Generative Expressive Conversational Speech Synthesis Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In addition, due to the limitations of small-scale datasets containing scripted recording styles, they often fail to simulate real natural conversational styles. To address the above issues, we propose a novel generative expressive CSS system, termed GPT-Talker. |
Rui Liu; Yifan Hu; Yi Ren; Xiang Yin; Haizhou Li; | arxiv-cs.CL | 2024-07-31 |
| 312 | Performance of Recent Large Language Models for A Low-Resourced Language Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Large Language Models (LLMs) have shown significant advances in the past year. |
Ravindu Jayakody; Gihan Dias; | arxiv-cs.CL | 2024-07-31 |
| 313 | Automated Software Vulnerability Static Code Analysis Using Generative Pre-Trained Transformer Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Ultimately, we find that the GPT models that we evaluated are not suitable for fully automated vulnerability scanning because the false positive and false negative rates are too high to likely be useful in practice. |
Elijah Pelofske; Vincent Urias; Lorie M. Liebrock; | arxiv-cs.CR | 2024-07-31 |
| 314 | Enhancing Agricultural Machinery Management Through Advanced LLM Integration Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper introduces a novel approach that leverages large language models (LLMs), particularly GPT-4, combined with multi-round prompt engineering to enhance decision-making processes in agricultural machinery management. |
Emily Johnson; Noah Wilson; | arxiv-cs.CL | 2024-07-30 |
| 315 | Robust Load Prediction of Power Network Clusters Based on Cloud-Model-Improved Transformer Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Presenting an innovative approach, the Cloud Model Improved Transformer (CMIT) method integrates the Transformer model with the cloud model utilizing the particle swarm optimization algorithm, with the aim of achieving robust and precise power load predictions. |
Cheng Jiang; Gang Lu; Xue Ma; Di Wu; | arxiv-cs.LG | 2024-07-30 |
| 316 | Interpretable Pre-Trained Transformers for Heart Time-Series Data Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we employ this framework to the analysis of clinical heart time-series data, to create two pre-trained general purpose cardiac models, termed PPG-PT and ECG-PT. |
Harry J. Davies; James Monsen; Danilo P. Mandic; | arxiv-cs.LG | 2024-07-30 |
| 317 | Comparison of Large Language Models for Generating Contextually Relevant Questions Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The contribution of this research is the analysis of the capacity of LLMs for Automatic Question Generation in education. |
IVO LODOVICO MOLINA et. al. | arxiv-cs.CL | 2024-07-30 |
| 318 | Legal Minds, Algorithmic Decisions: How LLMs Apply Constitutional Principles in Complex Scenarios Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we conduct an empirical analysis of how large language models (LLMs), specifically GPT-4, interpret constitutional principles in complex decision-making scenarios. |
Camilla Bignotti; Carolina Camassa; | arxiv-cs.CL | 2024-07-29 |
| 319 | Sentiment Analysis of Lithuanian Online Reviews Using Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we address sentiment analysis of Lithuanian five-star-based online reviews from multiple domains that we collect and clean. |
Brigita Vileikytė; Mantas Lukoševičius; Lukas Stankevičius; | arxiv-cs.CL | 2024-07-29 |
| 320 | DuA: Dual Attentive Transformer in Long-Term Continuous EEG Emotion Analysis Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, these methods encounter significant challenges in real-life scenarios where emotional states evolve over extended periods. To address this issue, we propose a Dual Attentive (DuA) transformer framework for long-term continuous EEG emotion analysis. |
YUE PAN et. al. | arxiv-cs.HC | 2024-07-29 |
| 321 | Survey and Taxonomy: The Role of Data-Centric AI in Transformer-Based Time Series Forecasting Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, there is a gap regarding the integration of transformer-based TSF and data-centric AI. This survey aims to pin down this gap via the extensive literature review based on the proposed taxonomy. |
Jingjing Xu; Caesar Wu; Yuan-Fang Li; Gregoire Danoy; Pascal Bouvry; | arxiv-cs.LG | 2024-07-29 |
| 322 | Motamot: A Dataset for Revealing The Supremacy of Large Language Models Over Transformer Models in Bengali Political Sentiment Analysis Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this study, we investigate political sentiment analysis during Bangladeshi elections, specifically examining how effectively Pre-trained Language Models (PLMs) and Large Language Models (LLMs) capture complex sentiment characteristics. |
FATEMA TUJ JOHORA FARIA et. al. | arxiv-cs.CL | 2024-07-28 |
| 323 | FarSSiBERT: A Novel Transformer-based Model for Semantic Similarity Measurement of Persian Social Networks Informal Texts Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper introduces a new transformer-based model to measure semantic similarity between Persian informal short texts from social networks. |
Seyed Mojtaba Sadjadi; Zeinab Rajabi; Leila Rabiei; Mohammad-Shahram Moin; | arxiv-cs.CL | 2024-07-27 |
| 324 | The Impact of LoRA Adapters for LLMs on Clinical NLP Classification Under Data Limitations Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Fine-tuning Large Language Models (LLMs) for clinical Natural Language Processing (NLP) poses significant challenges due to the domain gap and limited data availability. |
Thanh-Dung Le; Ti Ti Nguyen; Vu Nguyen Ha; | arxiv-cs.CL | 2024-07-27 |
| 325 | GPT Deciphering Fedspeak: Quantifying Dissent Among Hawks and Doves Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We use GPT-4 to quantify dissent among members on the topic of inflation. |
DENIS PESKOFF et. al. | arxiv-cs.AI | 2024-07-26 |
| 326 | QT-TDM: Planning with Transformer Dynamics Model and Autoregressive Q-Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Inspired by the success of the Transformer architecture in natural language processing and computer vision, we investigate the use of Transformers in Reinforcement Learning (RL), specifically in modeling the environment’s dynamics using Transformer Dynamics Models (TDMs). |
Mostafa Kotb; Cornelius Weber; Muhammad Burhan Hafez; Stefan Wermter; | arxiv-cs.LG | 2024-07-26 |
| 327 | Is Larger Always Better? Evaluating and Prompting Large Language Models for Non-generative Medical Tasks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This study benchmarks various models, including GPT-based LLMs, BERT-based models, and traditional clinical predictive models, for non-generative medical tasks utilizing renowned datasets. |
YINGHAO ZHU et. al. | arxiv-cs.CL | 2024-07-26 |
| 328 | Using GPT-4 to Guide Causal Machine Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we are interested in the ability of LLMs to identify causal relationships. |
Anthony C. Constantinou; Neville K. Kitson; Alessio Zanga; | arxiv-cs.AI | 2024-07-26 |
| 329 | Automatic Detection of Moral Values in Music Lyrics Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Moral values play a fundamental role in how we evaluate information, make decisions, and form judgements around important social issues. |
Vjosa Preniqi; Iacopo Ghinassi; Julia Ive; Kyriaki Kalimeri; Charalampos Saitis; | arxiv-cs.CY | 2024-07-26 |
| 330 | Gene Regulatory Network Inference from Pre-trained Single-Cell Transcriptomics Transformer with Joint Graph Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce a novel joint graph learning approach that combines the rich contextual representations learned by pre-trained single-cell language models with the structured knowledge encoded in GRNs using graph neural networks (GNNs). |
Sindhura Kommu; Yizhi Wang; Yue Wang; Xuan Wang; | arxiv-cs.LG | 2024-07-25 |
| 331 | HDL-GPT: High-Quality HDL Is All You Need Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents Hardware Description Language Generative Pre-trained Transformers (HDL-GPT), a novel approach that leverages the vast repository of open-source High Definition Language (HDL) codes to train superior quality large code models. |
BHUVNESH KUMAR et. al. | arxiv-cs.LG | 2024-07-25 |
| 332 | The Power of Combining Data and Knowledge: GPT-4o Is An Effective Interpreter of Machine Learning Models in Predicting Lymph Node Metastasis of Lung Cancer Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose a novel ensemble method that combines the medical knowledge acquired by LLMs with the latent patterns identified by machine learning models to enhance LNM prediction performance. |
Danqing Hu; Bing Liu; Xiaofeng Zhu; Nan Wu; | arxiv-cs.CL | 2024-07-25 |
| 333 | My Ontologist: Evaluating BFO-Based AI for Definition Support Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Through iterative development of a specialized GPT model named My Ontologist, we aimed to generate BFO-conformant ontologies. |
Carter Benson; Alec Sculley; Austin Liebers; John Beverley; | arxiv-cs.DB | 2024-07-24 |
| 334 | Testing Large Language Models on Driving Theory Knowledge and Skills for Connected Autonomous Vehicles Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we investigate a new approach of applying remote or edge LLMs to support autonomous driving. |
Zuoyin Tang; Jianhua He; Dashuai Pei; Kezhong Liu; Tao Gao; | arxiv-cs.AI | 2024-07-24 |
| 335 | Cost-effective Instruction Learning for Pathology Vision and Language Analysis Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Here we propose a cost-effective instruction learning framework for conversational pathology named as CLOVER. |
KAITAO CHEN et. al. | arxiv-cs.AI | 2024-07-24 |
| 336 | SDoH-GPT: Using Large Language Models to Extract Social Determinants of Health (SDoH) Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study we introduced SDoH-GPT, a simple and effective few-shot Large Language Model (LLM) method leveraging contrastive examples and concise instructions to extract SDoH without relying on extensive medical annotations or costly human intervention. |
BERNARDO CONSOLI et. al. | arxiv-cs.CL | 2024-07-24 |
| 337 | Artificial Intelligence in Extracting Diagnostic Data from Dental Records Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This research addresses the issue of missing structured data in dental records by extracting diagnostic information from unstructured text. |
YAO-SHUN CHUANG et. al. | arxiv-cs.CL | 2024-07-23 |
| 338 | Can Large Language Models Automatically Jailbreak GPT-4V? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In our study, we introduce AutoJailbreak, an innovative automatic jailbreak technique inspired by prompt optimization. |
YUANWEI WU et. al. | arxiv-cs.CL | 2024-07-23 |
| 339 | OriGen:Enhancing RTL Code Generation with Code-to-Code Augmentation and Self-Reflection Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: While open-source LLMs offer solutions to these concerns, they typically underperform commercial models in RTL code generation tasks, primarily due to the scarcity of high-quality open-source RTL datasets. To address this challenge, we introduce OriGen , a fully open-source framework that incorporates self-reflection capabilities and a novel dataset augmentation methodology for generating high-quality, large-scale RTL code. |
FAN CUI et. al. | arxiv-cs.AR | 2024-07-23 |
| 340 | KWT-Tiny: RISC-V Accelerated, Embedded Keyword Spotting Transformer Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper explores the adaptation of Transformerbased models for edge devices through the quantisation and hardware acceleration of the ARM Keyword Transformer (KWT) model on a RISC-V platform. |
Aness Al-Qawlaq; Ajay Kumar M; Deepu John; | arxiv-cs.AR | 2024-07-22 |
| 341 | Inverted Activations: Reducing Memory Footprint in Neural Network Training Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose a modification to the handling of activation tensors in pointwise nonlinearity layers. |
Georgii Novikov; Ivan Oseledets; | arxiv-cs.LG | 2024-07-22 |
| 342 | RadioRAG: Factual Large Language Models for Enhanced Diagnostics in Radiology Using Dynamic Retrieval Augmented Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Large language models (LLMs) have advanced the field of artificial intelligence (AI) in medicine. |
SOROOSH TAYEBI ARASTEH et. al. | arxiv-cs.CL | 2024-07-22 |
| 343 | Can GPT-4 Learn to Analyze Moves in Research Article Abstracts? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper we employ the affordances of GPT-4 to automate the annotation process by using natural language prompts. |
Danni Yu; Marina Bondi; Ken Hyland; | arxiv-cs.CL | 2024-07-22 |
| 344 | Dissecting Multiplication in Transformers: Insights Into LLMs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Based on observation and analysis, we infer the reasons of transformers deficiencies in multiplication tasks lies in their difficulty in calculating successive carryovers and caching intermediate results, and confirmed this inference through experiments. Guided by these findings, we propose improvements to enhance transformers performance on multiplication tasks. |
Luyu Qiu; Jianing Li; Chi Su; Chen Jason Zhang; Lei Chen; | arxiv-cs.CL | 2024-07-22 |
| 345 | Efficient Visual Transformer By Learnable Token Merging Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose a novel and compact transformer block, Transformer with Learnable Token Merging (LTM), or LTM-Transformer. |
Yancheng Wang; Yingzhen Yang; | arxiv-cs.CV | 2024-07-21 |
| 346 | Unipa-GPT: Large Language Models for University-oriented QA in Italian Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In our experiments we adopted both the Retrieval Augmented Generation (RAG) approach and fine-tuning to develop the system. |
Irene Siragusa; Roberto Pirrone; | arxiv-cs.CL | 2024-07-19 |
| 347 | LLMs Left, Right, and Center: Assessing GPT’s Capabilities to Label Political Bias from Web Domains Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Given the subjective nature of political labels, third-party bias ratings like those from Ad Fontes Media, AllSides, and Media Bias/Fact Check (MBFC) are often used in research to analyze news source diversity. This study aims to determine if GPT-4 can replicate these human ratings on a seven-degree scale (far-left to far-right). |
Raphael Hernandes; | arxiv-cs.CL | 2024-07-19 |
| 348 | Adaptive Foundation Models for Online Decisions: HyperAgent with Fast Incremental Uncertainty Estimation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce GPT-HyperAgent, an augmentation of GPT with HyperAgent for uncertainty-aware, scalable exploration in contextual bandits, a fundamental online decision problem involving natural language input. |
Yingru Li; Jiawei Xu; Zhi-Quan Luo; | arxiv-cs.LG | 2024-07-18 |
| 349 | Can Open-Source LLMs Compete with Commercial Models? Exploring The Few-Shot Performance of Current GPT Models in Biomedical Tasks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We participated in the 12th BioASQ challenge, which is a retrieval augmented generation (RAG) setting, and explored the performance of current GPT models Claude 3 Opus, GPT-3.5-turbo and Mixtral 8x7b with in-context learning (zero-shot, few-shot) and QLoRa fine-tuning. |
Samy Ateia; Udo Kruschwitz; | arxiv-cs.CL | 2024-07-18 |
| 350 | Evaluating Large Language Models for Anxiety and Depression Classification Using Counseling and Psychotherapy Transcripts Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We aim to evaluate the efficacy of traditional machine learning and large language models (LLMs) in classifying anxiety and depression from long conversational transcripts. |
Junwei Sun; Siqi Ma; Yiran Fan; Peter Washington; | arxiv-cs.CL | 2024-07-18 |
| 351 | A Light-weight and Efficient Punctuation and Word Casing Prediction Model for On-device Streaming ASR Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose a light-weight and efficient model that jointly predicts punctuation and word casing in real time. |
Jian You; Xiangfeng Li; | arxiv-cs.CL | 2024-07-18 |
| 352 | ARTEMIS: A Mixed Analog-Stochastic In-DRAM Accelerator for Transformer Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose ARTEMIS, a mixed analog-stochastic in-DRAM accelerator for transformer models. |
Salma Afifi; Ishan Thakkar; Sudeep Pasricha; | arxiv-cs.AR | 2024-07-17 |
| 353 | Sharif-STR at SemEval-2024 Task 1: Transformer As A Regression Model for Fine-Grained Scoring of Textual Semantic Relations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we delve into the investigation of sentence-level STR within Track A (Supervised) by leveraging fine-tuning techniques on the RoBERTa transformer. |
SEYEDEH FATEMEH EBRAHIMI et. al. | arxiv-cs.CL | 2024-07-17 |
| 354 | Frequency Guidance Matters: Skeletal Action Recognition By Frequency-Aware Mixed Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, the existing transformer-based approaches heavily rely on the naive attention mechanism for capturing the spatiotemporal features, which falls short in learning discriminative representations that exhibit similar motion patterns. To address this challenge, we introduce the Frequency-aware Mixed Transformer (FreqMixFormer), specifically designed for recognizing similar skeletal actions with subtle discriminative motions. |
WENHAN WU et. al. | arxiv-cs.CV | 2024-07-17 |
| 355 | LLMs-in-the-loop Part-1: Expert Small AI Models for Bio-Medical Text Translation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study introduces a novel LLMs-in-the-loop approach to develop supervised neural machine translation models optimized specifically for medical texts. |
Bunyamin Keles; Murat Gunay; Serdar I. Caglar; | arxiv-cs.CL | 2024-07-16 |
| 356 | Large Language Models As Misleading Assistants in Conversation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We investigate the ability of LLMs to be deceptive in the context of providing assistance on a reading comprehension task, using LLMs as proxies for human users. |
BETTY LI HOU et. al. | arxiv-cs.CL | 2024-07-16 |
| 357 | Educational Personalized Learning Path Planning with Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Despite its potential, traditional PLPP systems often lack adaptability, interactivity, and transparency. This paper proposes a novel approach integrating Large Language Models (LLMs) with prompt engineering to address these challenges. |
Chee Ng; Yuen Fung; | arxiv-cs.CL | 2024-07-16 |
| 358 | Does Refusal Training in LLMs Generalize to The Past Tense? Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We systematically evaluate this method on Llama-3 8B, Claude-3.5 Sonnet, GPT-3.5 Turbo, Gemma-2 9B, Phi-3-Mini, GPT-4o mini, GPT-4o, o1-mini, o1-preview, and R2D2 models using GPT-3.5 Turbo as a reformulation model. |
Maksym Andriushchenko; Nicolas Flammarion; | arxiv-cs.CL | 2024-07-16 |
| 359 | A Transformer-based Approach for Augmenting Software Engineering Chatbots Datasets Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, previous studies show that creating a high-quality training dataset for software engineering chatbots is expensive in terms of both resources and time. Aims: Therefore, in this paper, we present an automated transformer-based approach to augment software engineering chatbot datasets. |
Ahmad Abdellatif; Khaled Badran; Diego Elias Costa; Emad Shihab; | arxiv-cs.SE | 2024-07-16 |
| 360 | Rethinking Transformer-based Multi-document Summarization: An Empirical Investigation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To thoroughly examine the behaviours of Transformer-based MDS models, this paper presents five empirical studies on (1) measuring the impact of document boundary separators quantitatively; (2) exploring the effectiveness of different mainstream Transformer structures; (3) examining the sensitivity of the encoder and decoder; (4) discussing different training strategies; and (5) discovering the repetition in a summary generation. |
Congbo Ma; Wei Emma Zhang; Dileepa Pitawela; Haojie Zhuang; Yanfeng Shu; | arxiv-cs.CL | 2024-07-16 |
| 361 | GPT Assisted Annotation of Rhetorical and Linguistic Features for Interpretable Propaganda Technique Detection in News Text Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Our contribution is a set of features, their properties, definitions, and examples in a machine-readable format, along with the code for RhetAnn and the GPT prompts and fine-tuning procedures for advancing state-of-the-art interpretable propaganda technique detection. |
Kyle Hamilton; Luca Longo; Bojan Bozic; | arxiv-cs.CL | 2024-07-16 |
| 362 | ECoh: Turn-level Coherence Evaluation for Multilingual Dialogues Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Motivated by the need for lightweight, open source, and multilingual dialogue evaluators, this paper introduces GenResCoh (Generated Responses targeting Coherence). |
John Mendonça; Isabel Trancoso; Alon Lavie; | arxiv-cs.CL | 2024-07-16 |
| 363 | Sharif-MGTD at SemEval-2024 Task 8: A Transformer-Based Approach to Detect Machine Generated Text Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this research, we explore the effectiveness of fine-tuning a RoBERTa-base transformer, a powerful neural architecture, to address MGT detection as a binary classification task. |
SEYEDEH FATEMEH EBRAHIMI et. al. | arxiv-cs.CL | 2024-07-16 |
| 364 | GPT-4V Cannot Generate Radiology Reports Yet Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we perform a systematic evaluation of GPT-4V in generating radiology reports on two chest X-ray report datasets: MIMIC-CXR and IU X-Ray. |
Yuyang Jiang; Chacha Chen; Dang Nguyen; Benjamin M. Mervak; Chenhao Tan; | arxiv-cs.CY | 2024-07-16 |
| 365 | R-SFLLM: Jamming Resilient Framework for Split Federated Learning with Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, rigorous insights are provided into the influence of jamming LLM word embeddings in SFL by deriving an expression for the ML training loss divergence and showing that it is upper-bounded by the mean squared error (MSE). |
ALADIN DJUHERA et. al. | arxiv-cs.LG | 2024-07-16 |
| 366 | Scientific QA System with Verifiable Answers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce the VerifAI project, a pioneering open-source scientific question-answering system, designed to provide answers that are not only referenced but also automatically vetted and verifiable. |
ADELA LJAJIĆ et. al. | arxiv-cs.CL | 2024-07-16 |
| 367 | GPT Sonograpy: Hand Gesture Decoding from Forearm Ultrasound Images Via VLM Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We show that GPT-4o can decode hand gestures from forearm ultrasound data even with no fine-tuning, and improves with few-shot, in-context learning. |
Keshav Bimbraw; Ye Wang; Jing Liu; Toshiaki Koike-Akino; | arxiv-cs.CV | 2024-07-15 |
| 368 | Beyond Binary: Multiclass Paraphasia Detection with Generative Pretrained Transformers and End-to-End Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present novel approaches that use a generative pretrained transformer (GPT) to identify paraphasias from transcripts as well as two end-to-end approaches that focus on modeling both automatic speech recognition (ASR) and paraphasia classification as multiple sequences vs. a single sequence. |
Matthew Perez; Aneesha Sampath; Minxue Niu; Emily Mower Provost; | arxiv-cs.CL | 2024-07-15 |
| 369 | Transformer-based Drum-level Prediction in A Boiler Plant with Delayed Relations Among Multivariates Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Leveraging the capabilities of Transformer architectures, this study aims to develop an accurate and robust predictive framework to anticipate water level fluctuations and facilitate proactive control strategies. |
Gang Su; Sun Yang; Zhishuai Li; | arxiv-cs.LG | 2024-07-15 |
| 370 | Leveraging LLM-Respondents for Item Evaluation: A Psychometric Analysis Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, item calibration is time-consuming and costly, requiring a sufficient number of respondents for the response process. We explore using six different LLMs (GPT-3.5, GPT-4, Llama 2, Llama 3, Gemini-Pro, and Cohere Command R Plus) and various combinations of them using sampling methods to produce responses with psychometric properties similar to human answers. |
Yunting Liu; Shreya Bhandari; Zachary A. Pardos; | arxiv-cs.CY | 2024-07-15 |
| 371 | DistillSeq: A Framework for Safety Alignment Testing in Large Language Models Using Knowledge Distillation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Subsequently, we deploy two distinct strategies for generating malicious queries: one based on a syntax tree approach, and the other leveraging an LLM-based method. |
Mingke Yang; Yuqi Chen; Yi Liu; Ling Shi; | arxiv-cs.SE | 2024-07-14 |
| 372 | CodeV: Empowering LLMs for Verilog Generation Through Multi-Level Summarization Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: (2) LLMs like GPT-3.5 excel in summarizing Verilog code rather than generating it. Based on these observations, this paper introduces CodeV, a series of open-source instruction-tuned Verilog generation LLMs. |
YANG ZHAO et. al. | arxiv-cs.PL | 2024-07-14 |
| 373 | Enhancing Emotion Prediction in News Headlines: Insights from ChatGPT and Seq2Seq Models for Free-Text Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The free-text explanations also contain more sentimental context than the news headlines alone and can serve as a better input to emotion classification models. Therefore, in this work we explored generating emotion explanations from headlines by training a sequence-to-sequence transformer model and by using pretrained large language model, ChatGPT (GPT-4). |
GE GAO et. al. | arxiv-cs.CL | 2024-07-14 |
| 374 | Graph Transformers: A Survey Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Beyond technical analysis, we discuss the applications of graph transformer models for node-level, edge-level, and graph-level tasks, exploring their potential in other application scenarios as well. |
AHSAN SHEHZAD et. al. | arxiv-cs.LG | 2024-07-13 |
| 375 | Causality Extraction from Medical Text Using Large Language Models (LLMs) Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study explores the potential of natural language models, including large language models, to extract causal relations from medical texts, specifically from Clinical Practice Guidelines (CPGs). |
Seethalakshmi Gopalakrishnan; Luciana Garbayo; Wlodek Zadrozny; | arxiv-cs.CL | 2024-07-13 |
| 376 | Document-level Clinical Entity and Relation Extraction Via Knowledge Base-Guided Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Generative pre-trained transformer (GPT) models have shown promise in clinical entity and relation extraction tasks because of their precise extraction and contextual understanding capability. |
Kriti Bhattarai; Inez Y. Oh; Zachary B. Abrams; Albert M. Lai; | arxiv-cs.CL | 2024-07-13 |
| 377 | Robustness of LLMs to Perturbations in Text Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To that end, we artificially introduce varying levels of noise into a diverse set of datasets and systematically evaluate LLMs’ robustness against the corrupt variations of the original text. |
Ayush Singh; Navpreet Singh; Shubham Vatsal; | arxiv-cs.CL | 2024-07-12 |
| 378 | EVOLVE: Predicting User Evolution and Network Dynamics in Social Media Using Fine-Tuned GPT-like Model Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this research, we propose a predictive method to understand how a user evolves on social media throughout their life and to forecast the next stage of their evolution. |
Ismail Hossain; Md Jahangir Alam; Sai Puppala; Sajedul Talukder; | arxiv-cs.SI | 2024-07-12 |
| 379 | ASTPrompter: Weakly Supervised Automated Language Model Red-Teaming to Identify Likely Toxic Prompts Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Here, we propose a reinforcement learning formulation of the LLM red-teaming task which allows us to discover prompts that both (1) trigger toxic outputs from a frozen defender and (2) have low perplexity as scored by the defender. |
Amelia F. Hardy; Houjun Liu; Bernard Lange; Mykel J. Kochenderfer; | arxiv-cs.CL | 2024-07-12 |
| 380 | Movie Recommendation with Poster Attention Via Multi-modal Transformer Feature Fusion Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study proposes a multi-modal movie recommendation system by extract features of the well designed posters for each movie and the narrative text description of the movie. |
Linhan Xia; Yicheng Yang; Ziou Chen; Zheng Yang; Shengxin Zhu; | arxiv-cs.IR | 2024-07-12 |
| 381 | The Two Sides of The Coin: Hallucination Generation and Detection with LLMs As Evaluators for LLMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This work presents our participation in the CLEF ELOQUENT HalluciGen shared task, where the goal is to develop evaluators for both generating and detecting hallucinated content. |
ANH THU MARIA BUI et. al. | arxiv-cs.AI | 2024-07-12 |
| 382 | On Exact Bit-level Reversible Transformers Without Changing Architectures Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work we present the BDIA-transformer, which is an exact bit-level reversible transformer that uses an unchanged standard architecture for inference. |
Guoqiang Zhang; J. P. Lewis; W. B. Kleijn; | arxiv-cs.LG | 2024-07-12 |
| 383 | Show, Don’t Tell: Evaluating Large Language Models Beyond Textual Understanding with ChildPlay Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To evaluate the models’ ability to generalize beyond their training data, we introduce two additional games. |
Gonçalo Hora de Carvalho; Oscar Knap; Robert Pollice; | arxiv-cs.AI | 2024-07-12 |
| 384 | Detect Llama — Finding Vulnerabilities in Smart Contracts Using Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we test the hypothesis that although OpenAI’s GPT-4 performs well generally, we can fine-tune open-source models to outperform GPT-4 in smart contract vulnerability detection. |
Peter Ince; Xiapu Luo; Jiangshan Yu; Joseph K. Liu; Xiaoning Du; | arxiv-cs.CR | 2024-07-11 |
| 385 | LLMs’ Morphological Analyses of Complex FST-generated Finnish Words Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This work aims to shed light on the issue by evaluating state-of-the-art LLMs in a task of morphological analysis of complex Finnish noun forms. |
Anssi Moisio; Mathias Creutz; Mikko Kurimo; | arxiv-cs.CL | 2024-07-11 |
| 386 | GPT-4 Is Judged More Human Than Humans in Displaced and Inverted Turing Tests Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We found that both AI and displaced human judges were less accurate than interactive interrogators, with below chance accuracy overall. |
Ishika Rathi; Sydney Taylor; Benjamin K. Bergen; Cameron R. Jones; | arxiv-cs.HC | 2024-07-11 |
| 387 | Teaching Transformers Causal Reasoning Through Axiomatic Training Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Since interventional data is costly to generate, we study to what extent an agent can learn causal reasoning from passive data. |
Aniket Vashishtha; Abhinav Kumar; Abbavaram Gowtham Reddy; Vineeth N Balasubramanian; Amit Sharma; | arxiv-cs.LG | 2024-07-10 |
| 388 | FsPONER: Few-shot Prompt Optimization for Named Entity Recognition in Domain-specific Scenarios Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, none of the previous approaches has investigated the efficiency of LLM-based few-shot learning in domain-specific scenarios. To address this gap, we introduce FsPONER, a novel approach for optimizing few-shot prompts, and evaluate its performance on domain-specific NER datasets, with a focus on industrial manufacturing and maintenance, while using multiple LLMs — GPT-4-32K, GPT-3.5-Turbo, LLaMA 2-chat, and Vicuna. |
Yongjian Tang; Rakebul Hasan; Thomas Runkler; | arxiv-cs.CL | 2024-07-10 |
| 389 | ROSA: Random Subspace Adaptation for Efficient Fine-Tuning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work we propose Random Subspace Adaptation (ROSA), a method that outperforms previous PEFT methods by a significant margin, while maintaining a zero latency overhead during inference time. |
Marawan Gamal Abdel Hameed; Aristides Milios; Siva Reddy; Guillaume Rabusseau; | arxiv-cs.LG | 2024-07-10 |
| 390 | Prompting Techniques for Secure Code Generation: A Systematic Investigation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: OBJECTIVE: In this study, we investigate the impact of different prompting techniques on the security of code generated from NL instructions by LLMs. |
Catherine Tony; Nicolás E. Díaz Ferreyra; Markus Mutas; Salem Dhiff; Riccardo Scandariato; | arxiv-cs.SE | 2024-07-09 |
| 391 | Mixture-of-Modules: Reinventing Transformers As Dynamic Assemblies of Modules Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We show that MoM provides not only a unified framework for Transformers and their numerous variants but also a flexible and learnable approach for reducing redundancy in Transformer parameterization. |
ZHUOCHENG GONG et. al. | arxiv-cs.CL | 2024-07-09 |
| 392 | A Comparison of Vulnerability Feature Extraction Methods from Textual Attack Patterns Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we examine five feature extraction methods (TF-IDF, LSI, BERT, MiniLM, RoBERTa) and find that Term Frequency-Inverse Document Frequency (TF-IDF) outperforms the other four methods with a precision of 75\% and an F1 score of 64\%. |
Refat Othman; Bruno Rossi; Russo Barbara; | arxiv-cs.CR | 2024-07-09 |
| 393 | Using Large Language Models for Generating Smart Contracts for Health Insurance from Textual Policies Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To assess the LLM output, we propose completeness, soundness, clarity, syntax, and functioning code as metrics. |
Inwon Kang; William Van Woensel; Oshani Seneviratne; | arxiv-cs.CL | 2024-07-09 |
| 394 | Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, we introduce Multilingual Blending, a mixed-language query-response scheme designed to evaluate the safety alignment of various state-of-the-art LLMs (e.g., GPT-4o, GPT-3.5, Llama3) under sophisticated, multilingual conditions. |
Jiayang Song; Yuheng Huang; Zhehua Zhou; Lei Ma; | arxiv-cs.CL | 2024-07-09 |
| 395 | Short Answer Scoring with GPT-4 Related Papers Related Patents Related Grants Related Venues Related Experts View |
Lan Jiang; Nigel Bosch; | ACM Conference on Learning @ Scale | 2024-07-09 |
| 396 | PEER: Expertizing Domain-Specific Tasks with A Multi-Agent Framework and Tuning Methods Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: High performance requires sophisticated processing techniques, yet managing multiple agents within a complex workflow often proves costly and challenging. To address this, we introduce the PEER (Plan, Execute, Express, Review) multi-agent framework. |
YIYING WANG et. al. | arxiv-cs.AI | 2024-07-09 |
| 397 | Cross-domain Few-shot In-context Learning for Enhancing Traffic Sign Recognition Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose a cross-domain few-shot in-context learning method based on the MLLM for enhancing traffic sign recognition (TSR). |
YAOZONG GAN et. al. | arxiv-cs.CV | 2024-07-08 |
| 398 | On The Power of Convolution Augmented Transformer Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, recent architectural recipes, such as state-space models, have bridged the performance gap. Motivated by this, we examine the benefits of Convolution-Augmented Transformer (CAT) for recall, copying, and length generalization tasks. |
Mingchen Li; Xuechen Zhang; Yixiao Huang; Samet Oymak; | arxiv-cs.LG | 2024-07-08 |
| 399 | Surprising Gender Biases in GPT Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present seven experiments exploring gender biases in GPT. |
Raluca Alexandra Fulgu; Valerio Capraro; | arxiv-cs.CY | 2024-07-08 |
| 400 | Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This work introduces the Multimodal Diffusion Transformer (MDT), a novel diffusion policy framework, that excels at learning versatile behavior from multimodal goal specifications with few language annotations. |
Moritz Reuss; Ömer Erdinç Yağmurlu; Fabian Wenzel; Rudolf Lioutikov; | arxiv-cs.RO | 2024-07-08 |
| 401 | Enhancing Computer Programming Education with LLMs: A Study on Effective Prompt Engineering for Python Code Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study underscores the crucial role of prompt engineering in maximizing the educational benefits of LLMs. By systematically categorizing and testing these strategies, we provide a comprehensive framework for both educators and students to optimize LLM-based learning experiences. |
Tianyu Wang; Nianjun Zhou; Zhixiong Chen; | arxiv-cs.AI | 2024-07-07 |
| 402 | Image-Conditional Diffusion Transformer for Underwater Image Enhancement Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Motivated by the recent advance in generative models, we propose a novel UIE method based on image-conditional diffusion transformer (ICDT). |
XINGYANG NIE et. al. | arxiv-cs.CV | 2024-07-07 |
| 403 | Associative Recurrent Memory Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper addresses the challenge of creating a neural architecture for very long sequences that requires constant time for processing new information at each time step. |
Ivan Rodkin; Yuri Kuratov; Aydar Bulatov; Mikhail Burtsev; | arxiv-cs.CL | 2024-07-05 |
| 404 | MMSci: A Multimodal Multi-Discipline Dataset for PhD-Level Scientific Comprehension Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Current datasets and benchmarks primarily focus on relatively simple scientific tasks and figures, lacking comprehensive assessments across diverse advanced scientific disciplines. To bridge this gap, we collected a multimodal, multidisciplinary dataset from open-access scientific articles published in Nature Communications journals. |
ZEKUN LI et. al. | arxiv-cs.CL | 2024-07-05 |
| 405 | Using LLMs to Label Medical Papers According to The CIViC Evidence Model Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce the sequence classification problem CIViC Evidence to the field of medical NLP. |
Markus Hisch; Xing David Wang; | arxiv-cs.CL | 2024-07-05 |
| 406 | Generalists Vs. Specialists: Evaluating Large Language Models for Urdu Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we compare general-purpose models, GPT-4-Turbo and Llama-3-8b, with special-purpose models–XLM-Roberta-large, mT5-large, and Llama-3-8b–that have been fine-tuned on specific tasks. |
Samee Arif; Abdul Hameed Azeemi; Agha Ali Raza; Awais Athar; | arxiv-cs.CL | 2024-07-05 |
| 407 | GPT Vs RETRO: Exploring The Intersection of Retrieval and Parameter-Efficient Fine-Tuning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we apply PEFT methods (P-tuning, Adapters, and LoRA) to a modified Retrieval-Enhanced Transformer (RETRO) and a baseline GPT model across several sizes, ranging from 823 million to 48 billion parameters. |
Aleksander Ficek; Jiaqi Zeng; Oleksii Kuchaiev; | arxiv-cs.CL | 2024-07-05 |
| 408 | Towards Automating Text Annotation: A Case Study on Semantic Proximity Annotation Using GPT-4 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The main aim of this paper is to reuse human annotation guidelines along with some annotated data to design automatic prompts for LLMs, focusing on the semantic proximity annotation task. |
Sachin Yadav; Tejaswi Choppa; Dominik Schlechtweg; | arxiv-cs.CL | 2024-07-04 |
| 409 | HYBRINFOX at CheckThat! 2024 — Task 2: Enriching BERT Models with The Expert System VAGO for Subjectivity Detection Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents the HYBRINFOX method used to solve Task 2 of Subjectivity detection of the CLEF 2024 CheckThat! |
MORGANE CASANOVA et. al. | arxiv-cs.CL | 2024-07-04 |
| 410 | From Data to Commonsense Reasoning: The Use of Large Language Models for Explainable AI Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we investigate the effectiveness of large language models (LLMs) on different QA tasks with a focus on their abilities in reasoning and explainability. |
Stefanie Krause; Frieder Stolzenburg; | arxiv-cs.AI | 2024-07-04 |
| 411 | TrackPGD: A White-box Attack Using Binary Masks Against Robust Transformer Trackers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We are proposing a novel white-box attack named TrackPGD, which relies on the predicted object binary mask to attack the robust transformer trackers. |
Fatemeh Nourilenjan Nokabadi; Yann Batiste Pequignot; Jean-Francois Lalonde; Christian Gagné; | arxiv-cs.CV | 2024-07-04 |
| 412 | Adaptive Step-size Perception Unfolding Network with Non-local Hybrid Attention for Hyperspectral Image Reconstruction Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To overcome the aforementioned drawbacks, We proposed an adaptive step-size perception unfolding network (ASPUN), a deep unfolding network based on FISTA algorithm, which uses an adaptive step-size perception module to estimate the update step-size of each spectral channel. |
Yanan Yang; Like Xin; | arxiv-cs.CV | 2024-07-04 |
| 413 | GPT-4 Vs. Human Translators: A Comprehensive Evaluation of Translation Quality Across Languages, Domains, and Expertise Levels Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study comprehensively evaluates the translation quality of Large Language Models (LLMs), specifically GPT-4, against human translators of varying expertise levels across multiple language pairs and domains. |
JIANHAO YAN et. al. | arxiv-cs.CL | 2024-07-04 |
| 414 | CATT: Character-based Arabic Tashkeel Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper introduces a new approach to training ATD models. |
Faris Alasmary; Orjuwan Zaafarani; Ahmad Ghannam; | arxiv-cs.CL | 2024-07-03 |
| 415 | Regurgitative Training: The Value of Real Data in Training Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: What happens if we train a new Large Language Model (LLM) using data that are at least partially generated by other LLMs? |
Jinghui Zhang; Dandan Qiao; Mochen Yang; Qiang Wei; | arxiv-cs.CL | 2024-07-03 |
| 416 | Mast Kalandar at SemEval-2024 Task 8: On The Trail of Textual Origins: RoBERTa-BiLSTM Approach to Detect AI-Generated Text Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: SemEval 2024 introduces the task of Multigenerator, Multidomain, and Multilingual Black-Box Machine-Generated Text Detection, aiming to develop automated systems for identifying machine-generated text and detecting potential misuse. In this paper, we i) propose a RoBERTa-BiLSTM based classifier designed to classify text into two categories: AI-generated or human ii) conduct a comparative study of our model with baseline approaches to evaluate its effectiveness. |
Jainit Sushil Bafna; Hardik Mittal; Suyash Sethia; Manish Shrivastava; Radhika Mamidi; | arxiv-cs.CL | 2024-07-03 |
| 417 | Large Language Models As Evaluators for Scientific Synthesis Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Our study explores how well the state-of-the-art Large Language Models (LLMs), like GPT-4 and Mistral, can assess the quality of scientific summaries or, more fittingly, scientific syntheses, comparing their evaluations to those of human annotators. |
Julia Evans; Jennifer D’Souza; Sören Auer; | arxiv-cs.CL | 2024-07-03 |
| 418 | InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present InternLM-XComposer-2.5 (IXC-2.5), a versatile large-vision language model that supports long-contextual input and output. |
PAN ZHANG et. al. | arxiv-cs.CV | 2024-07-03 |
| 419 | Assessing The Code Clone Detection Capability of Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study aims to assess the performance of two advanced Large Language Models (LLMs), GPT-3.5 and GPT-4, in the task of code clone detection. |
Zixian Zhang; Takfarinas Saber; | arxiv-cs.SE | 2024-07-02 |
| 420 | RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we propose a novel instruction fine-tuning framework RankRAG, which instruction-tunes a single LLM for the dual purpose of context ranking and answer generation in RAG. |
YUE YU et. al. | arxiv-cs.CL | 2024-07-02 |
| 421 | Image-to-Text Logic Jailbreak: Your Imagination Can Help You Do Anything Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: With the integration of visual and text inputs in VLMs, new security issues emerge, as malicious attackers can exploit multiple modalities to achieve their objectives. |
Xiaotian Zou; Ke Li; Yongkang Chen; | arxiv-cs.CR | 2024-07-01 |
| 422 | Hypformer: Exploring Efficient Hyperbolic Transformer Fully in Hyperbolic Space Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This stems primarily from: (i) the absence of well-defined modules in hyperbolic space, including linear transformation layers, LayerNorm layers, activation functions, dropout operations, etc. (ii) the quadratic time complexity of the existing hyperbolic self-attention module w.r.t the number of input tokens, which hinders its scalability. To address these challenges, we propose, Hypformer, a novel hyperbolic Transformer based on the Lorentz model of hyperbolic geometry. |
MENGLIN YANG et. al. | arxiv-cs.LG | 2024-07-01 |
| 423 | FATFusion: A Functional-anatomical Transformer for Medical Image Fusion Related Papers Related Patents Related Grants Related Venues Related Experts View |
Wei Tang; Fazhi He; | Inf. Process. Manag. | 2024-07-01 |
| 424 | Transformer Autoencoder for K-means Efficient Clustering Related Papers Related Patents Related Grants Related Venues Related Experts View |
Wenhao Wu; Weiwei Wang; Xixi Jia; Xiangchu Feng; | Eng. Appl. Artif. Intell. | 2024-07-01 |
| 425 | Beyond Numeric Awards: In-Context Dueling Bandits with LLM Agents Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Nevertheless, LLMs struggle to converge even when explicitly prompted to do so, and are sensitive to prompt variations. To overcome these issues, we introduce an LLM-augmented algorithm, IF-Enhanced LLM, which takes advantage of both in-context decision-making capabilities of LLMs and theoretical guarantees inherited from classic DB algorithms. |
Fanzeng Xia; Hao Liu; Yisong Yue; Tongxin Li; | arxiv-cs.LG | 2024-07-01 |
| 426 | MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizations Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This work presents MMLongBench-Doc, a long-context, multi-modal benchmark comprising 1,062 expert-annotated questions. |
YUBO MA et. al. | arxiv-cs.CV | 2024-07-01 |
| 427 | Analyzing Persuasive Strategies in Meme Texts: A Fusion of Language Models with Paraphrase Enrichment Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper describes our approach to hierarchical multi-label detection of persuasion techniques in meme texts. |
Kota Shamanth Ramanath Nayak; Leila Kosseim; | arxiv-cs.CL | 2024-07-01 |
| 428 | Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Despite the wide adoption of hybrid parallel paradigms like model parallelism, expert parallelism, and expert-sharding parallelism (i.e., MP+EP+ESP) to support MoE model training on GPU clusters, the training efficiency is hindered by communication costs introduced by these parallel paradigms. To address this limitation, we propose Parm, a system that accelerates MP+EP+ESP training by designing two dedicated schedules for placing communication tasks. |
XINGLIN PAN et. al. | arxiv-cs.DC | 2024-06-30 |
| 429 | LegalTurk Optimized BERT for Multi-Label Text Classification and NER Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we introduce our innovative modified pre-training approach by combining diverse masking strategies. |
Farnaz Zeidi; Mehmet Fatih Amasyali; Çiğdem Erol; | arxiv-cs.CL | 2024-06-30 |
| 430 | WallFacer: Harnessing Multi-dimensional Ring Parallelism for Efficient Long Sequence Model Training Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Current methods are either constrained by the number of attention heads or excessive communication overheads. To address this problem, we propose WallFacer, a multi-dimensional distributed training system for long sequences, fostering an efficient communication paradigm and providing additional tuning flexibility for communication arrangements. |
ZIMING LIU et. al. | arxiv-cs.DC | 2024-06-30 |
| 431 | LLM-Generated Natural Language Meets Scaling Laws: New Explorations and Data Augmentation Methods Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Nonetheless, prior research harbors two primary concerns: firstly, a lack of contemplation regarding whether the natural language generated by LLM (LLMNL) truly aligns with human natural language (HNL), a critical foundational question; secondly, an oversight that augmented data is randomly generated by LLM, implying that not all data may possess equal training value, that could impede the performance of classifiers. To address these challenges, we introduce the scaling laws to intrinsically calculate LLMNL and HNL. |
Zhenhua Wang; Guang Xu; Ming Ren; | arxiv-cs.CL | 2024-06-29 |
| 432 | Can GPT-4 Help Detect Quit Vaping Intentions? An Exploration of Automatic Data Annotation Approach Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, we extracted a sample dataset from one vaping sub-community on Reddit to analyze users’ quit-vaping intentions. |
SAI KRISHNA REVANTH VURUMA et. al. | arxiv-cs.CL | 2024-06-28 |
| 433 | Machine Learning Predictors for Min-Entropy Estimation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Utilizing data from Generalized Binary Autoregressive Models, a subset of Markov processes, we demonstrate that machine learning models (including a hybrid of convolutional and recurrent Long Short-Term Memory layers and the transformer-based GPT-2 model) outperform traditional NIST SP 800-90B predictors in certain scenarios. |
Javier Blanco-Romero; Vicente Lorenzo; Florina Almenares Mendoza; Daniel Díaz-Sánchez; | arxiv-cs.LG | 2024-06-28 |
| 434 | The Model Arena for Cross-lingual Sentiment Analysis: A Comparative Study in The Era of Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Sentiment analysis serves as a pivotal component in Natural Language Processing (NLP). |
Xiliang Zhu; Shayna Gardiner; Tere Roldán; David Rossouw; | arxiv-cs.CL | 2024-06-27 |
| 435 | Fine-tuned Network Relies on Generic Representation to Solve Unseen Cognitive Task Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Fine-tuning pretrained language models has shown promising results on a wide range of tasks, but when encountering a novel task, do they rely more on generic pretrained representation, or develop brand new task-specific solutions? |
Dongyan Lin; | arxiv-cs.LG | 2024-06-27 |
| 436 | FRED: Flexible REduction-Distribution Interconnect and Communication Implementation for Wafer-Scale Distributed Training of DNN Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose FRED, a wafer-scale interconnect that is tailored for the high-BW requirements of wafer-scale networks and can efficiently execute communication patterns of different parallelization strategies. |
Saeed Rashidi; William Won; Sudarshan Srinivasan; Puneet Gupta; Tushar Krishna; | arxiv-cs.AR | 2024-06-27 |
| 437 | NTFormer: A Composite Node Tokenized Graph Transformer for Node Classification Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To this end, we propose a new graph Transformer called NTFormer to address this issue. |
Jinsong Chen; Siyu Jiang; Kun He; | arxiv-cs.LG | 2024-06-27 |
| 438 | BADGE: BADminton Report Generation and Evaluation with LLM Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a novel framework named BADGE, designed for this purpose using LLM. |
Shang-Hsuan Chiang; Lin-Wei Chao; Kuang-Da Wang; Chih-Chuan Wang; Wen-Chih Peng; | arxiv-cs.CL | 2024-06-26 |
| 439 | Improving Entity Recognition Using Ensembles of Deep Learning and Fine-tuned Large Language Models: A Case Study on Adverse Event Extraction from Multiple Sources Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, we utilized reports and posts from the VAERS (n=621), Twitter (n=9,133), and Reddit (n=131) as our corpora. |
YIMING LI et. al. | arxiv-cs.CL | 2024-06-25 |
| 440 | SetBERT: Enhancing Retrieval Performance for Boolean Logic and Set Operation Queries Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce SetBERT, a fine-tuned BERT-based model designed to enhance query embeddings for set operations and Boolean logic queries, such as Intersection (AND), Difference (NOT), and Union (OR). |
Quan Mai; Susan Gauch; Douglas Adams; | arxiv-cs.CL | 2024-06-25 |
| 441 | This Paper Had The Smartest Reviewers — Flattery Detection Utilising An Audio-Textual Transformer-Based Approach Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Its automatic detection can thus enhance the naturalness of human-AI interactions. To meet this need, we present a novel audio textual dataset comprising 20 hours of speech and train machine learning models for automatic flattery detection. |
LUKAS CHRIST et. al. | arxiv-cs.SD | 2024-06-25 |
| 442 | CTBench: A Comprehensive Benchmark for Evaluating Language Model Capabilities in Clinical Trial Design Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: CTBench is introduced as a benchmark to assess language models (LMs) in aiding clinical study design. |
NAFIS NEEHAL et. al. | arxiv-cs.CL | 2024-06-25 |
| 443 | Unambiguous Recognition Should Not Rely Solely on Natural Language Training Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This bias stems from the inherent characteristics of the dataset. To mitigate this bias, we propose a LaTeX printed text recognition model trained on a mixed dataset of pseudo-formulas and pseudo-text. |
Renqing Luo; Yuhan Xu; | arxiv-cs.CV | 2024-06-24 |
| 444 | Exploring The Capability of Mamba in Speech Applications Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we compared Mamba with state-of-the-art Transformer variants for various speech applications, including ASR, text-to-speech, spoken language understanding, and speech summarization. |
Koichi Miyazaki; Yoshiki Masuyama; Masato Murata; | arxiv-cs.SD | 2024-06-24 |
| 445 | GPT-4V Explorations: Mining Autonomous Driving Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper explores the application of the GPT-4V(ision) large visual language model to autonomous driving in mining environments, where traditional systems often falter in understanding intentions and making accurate decisions during emergencies. |
Zixuan Li; | arxiv-cs.CV | 2024-06-24 |
| 446 | Exploring Factual Entailment with NLI: A News Media Study Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We explore the relationship between factuality and Natural Language Inference (NLI) by introducing FactRel — a novel annotation scheme that models \textit{factual} rather than \textit{textual} entailment, and use it to annotate a dataset of naturally occurring sentences from news articles. |
Guy Mor-Lan; Effi Levi; | arxiv-cs.CL | 2024-06-24 |
| 447 | DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we present DreamBench++, a human-aligned benchmark automated by advanced multimodal GPT models. |
YUANG PENG et. al. | arxiv-cs.CV | 2024-06-24 |
| 448 | The GPT-WritingPrompts Dataset: A Comparative Analysis of Character Portrayal in Short Stories Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We quantify and compare the emotional and descriptive features of storytelling from both generative processes, human and machine, along a set of six dimensions. |
Xi Yu Huang; Krishnapriya Vishnubhotla; Frank Rudzicz; | arxiv-cs.CL | 2024-06-24 |
| 449 | OlympicArena Medal Ranks: Who Is The Most Intelligent AI So Far? Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this report, we pose the following question: Who is the most intelligent AI model to date, as measured by the OlympicArena (an Olympic-level, multi-discipline, multi-modal benchmark for superintelligent AI)? |
Zhen Huang; Zengzhi Wang; Shijie Xia; Pengfei Liu; | arxiv-cs.CL | 2024-06-24 |
| 450 | CausalFormer: An Interpretable Transformer for Temporal Causal Discovery Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To facilitate the utilization of the whole deep learning models in temporal causal discovery, we proposed an interpretable transformer-based causal discovery model termed CausalFormer, which consists of the causality-aware transformer and the decomposition-based causality detector. |
LINGBAI KONG et. al. | arxiv-cs.LG | 2024-06-24 |
| 451 | GraphEval2000: Benchmarking and Improving Large Language Models on Graph Datasets Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, recent studies have identified limitations in LLMs’ ability to reason about graph-structured data. To address this gap, we introduce GraphEval2000, the first comprehensive graph dataset, comprising 40 graph data structure problems along with 2000 test cases. |
Qiming Wu; Zichen Chen; Will Corcoran; Misha Sra; Ambuj K. Singh; | arxiv-cs.AI | 2024-06-23 |
| 452 | Multi-Scale Temporal Difference Transformer for Video-Text Retrieval Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, they commonly neglect the inferior ability of the transformer modeling local temporal information. To tackle this problem, we propose a transformer variant named Multi-Scale Temporal Difference Transformer (MSTDT). |
Ni Wang; Dongliang Liao; Xing Xu; | arxiv-cs.CV | 2024-06-23 |
| 453 | Beyond Individual Facts: Investigating Categorical Knowledge Locality of Taxonomy and Meronomy Concepts in GPT Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we investigate a broader view of knowledge location, that of concepts or clusters of related information, instead of disparate individual facts. |
Christopher Burger; Yifan Hu; Thai Le; | arxiv-cs.LG | 2024-06-22 |
| 454 | How Effective Is GPT-4 Turbo in Generating School-Level Questions from Textbooks Based on Bloom’s Revised Taxonomy? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We evaluate the effectiveness of GPT-4 Turbo in generating educational questions from NCERT textbooks in zero-shot mode. |
Subhankar Maity; Aniket Deroy; Sudeshna Sarkar; | arxiv-cs.CL | 2024-06-21 |
| 455 | Toward Informal Language Processing: Knowledge of Slang in Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Using movie subtitles, we construct a dataset that supports evaluation on a diverse set of tasks pertaining to automatic processing of slang. |
Zhewei Sun; Qian Hu; Rahul Gupta; Richard Zemel; Yang Xu; | naacl | 2024-06-20 |
| 456 | VertAttack: Taking Advantage of Text Classifiers� Horizontal Vision Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Vertically written words willnot be recognized by a classifier. In contrast,humans are easily able to recognize and readwords written both horizontally and vertically.Hence, a human adversary could write problem-atic words vertically and the meaning wouldstill be preserved to other humans. We simulatesuch an attack, VertAttack. |
Jonathan Rusert; | naacl | 2024-06-20 |
| 457 | MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study aims to perform a thorough evaluation of the non-English capabilities of SoTA LLMs (GPT-3. |
SANCHIT AHUJA et. al. | naacl | 2024-06-20 |
| 458 | Unveiling Divergent Inductive Biases of LLMs on Temporal Data Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Despite the adeptness of Large Language Models (LLMs) in discerning patterns and relationships from data, their inherent comprehension of temporal dynamics remains a formidable challenge. This research meticulously explores these intrinsic challenges within LLMs, with a specific emphasis on evaluating the performance of GPT-3. |
Sindhu Kishore; Hangfeng He; | naacl | 2024-06-20 |
| 459 | Branch-Solve-Merge Improves Large Language Model Evaluation and Generation IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, their performance can fall short, due to the model�s lack of coherence and inability to plan and decompose the problem. We propose Branch-Solve-Merge (BSM), a Large Language Model program (Schlag et al. , 2023) for tackling such challenging natural language tasks. |
SWARNADEEP SAHA et. al. | naacl | 2024-06-20 |
| 460 | Transformers Can Represent N-gram Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we focus on the relationship between transformer LMs and n-gram LMs, a simple and historically relevant class of language models. |
Anej Svete; Ryan Cotterell; | naacl | 2024-06-20 |
| 461 | Communication-Efficient Byzantine-Resilient Federated Zero-Order Optimization Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce CYBER-0, the first zero-order optimization algorithm for memory-and-communication efficient Federated Learning, resilient to Byzantine faults. |
Afonso de Sá Delgado Neto; Maximilian Egger; Mayank Bakshi; Rawad Bitar; | arxiv-cs.LG | 2024-06-20 |
| 462 | A Symbolic Framework for Evaluating Mathematical Reasoning and Generalisation with Transformers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper proposes a methodology for generating and perturbing detailed derivations of equations at scale, aided by a symbolic engine, to evaluate the generalisability of Transformers to out-of-distribution mathematical reasoning problems. |
Jordan Meadows; Marco Valentino; Damien Teney; Andre Freitas; | naacl | 2024-06-20 |
| 463 | Does GPT Really Get It? A Hierarchical Scale to Quantify Human Vs AI’s Understanding of Algorithms Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We use the hierarchy to design and conduct a study with human subjects (undergraduate and graduate students) as well as large language models (generations of GPT), revealing interesting similarities and differences. |
Mirabel Reid; Santosh S. Vempala; | arxiv-cs.AI | 2024-06-20 |
| 464 | SemRoDe: Macro Adversarial Training to Learn Representations That Are Robust to Word-Level Attacks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose a novel approach called Semantic Robust Defence (SemRoDe), a Macro Adversarial Training strategy to enhance the robustness of LMs. |
Brian Formento; Wenjie Feng; Chuan-Sheng Foo; Anh Tuan Luu; See-Kiong Ng; | naacl | 2024-06-20 |
| 465 | Does GPT-4 Pass The Turing Test? IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: AI models with the ability to masquerade as humans could have widespread societal consequences, and we analyse the effectiveness of different strategies and criteria for judging humanlikeness. |
Cameron Jones; Ben Bergen; | naacl | 2024-06-20 |
| 466 | Metacognitive Prompting Improves Understanding in Large Language Models IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this study, we introduce Metacognitive Prompting (MP), a strategy inspired by human introspective reasoning processes. |
Yuqing Wang; Yun Zhao; | naacl | 2024-06-20 |
| 467 | Removing RLHF Protections in GPT-4 Via Fine-Tuning IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we show the contrary: fine-tuning allows attackers to remove RLHFprotections with as few as 340 examples and a 95% success rate. |
QIUSI ZHAN et. al. | naacl | 2024-06-20 |
| 468 | Struc-Bench: Are Large Language Models Good at Generating Complex Structured Tabular Data? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Our study assesses LLMs� proficiency in structuring tables and introduces a novel fine-tuning method, cognizant of data structures, to bolster their performance. |
XIANGRU TANG et. al. | naacl | 2024-06-20 |
| 469 | A Continued Pretrained LLM Approach for Automatic Medical Note Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce HEAL, the first continuously trained 13B LLaMA2-based LLM that is purpose-built for medical conversations and measured on automated scribing. |
DONG YUAN et. al. | naacl | 2024-06-20 |
| 470 | Revisiting Zero-Shot Abstractive Summarization in The Era of Large Language Models from The Perspective of Position Bias Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We characterize and study zero-shot abstractive summarization in Large Language Models (LLMs) by measuring position bias, which we propose as a general formulation of the more restrictive lead bias phenomenon studied previously in the literature. |
Anshuman Chhabra; Hadi Askari; Prasant Mohapatra; | naacl | 2024-06-20 |
| 471 | On Retrieval Augmentation and The Limitations of Language Model Training Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we rule out one previously posited possibility � the �softmax bottleneck. |
TING-RUI CHIANG et. al. | naacl | 2024-06-20 |
| 472 | CPopQA: Ranking Cultural Concept Popularity By LLMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, the extent to which an LLM effectively captures corpus-level statistical trends of concepts for reasoning, especially long-tail ones, is largely underexplored. In this study, we introduce a novel few-shot question-answering task (CPopQA) that examines LLMs� statistical ranking abilities for long-tail cultural concepts (e. g. , holidays), particularly focusing on these concepts� popularity in the United States and the United Kingdom, respectively. |
Ming Jiang; Mansi Joshi; | naacl | 2024-06-20 |
| 473 | SlimFit: Memory-Efficient Fine-Tuning of Transformer-based Models Using Training Dynamics Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, these models are extremely memory intensive during their fine-tuning process, making them difficult to deploy on GPUs with limited memory resources. To address this issue, we introduce a new tool called SlimFit that reduces the memory requirements of these models by dynamically analyzing their training dynamics and freezing less-contributory layers during fine-tuning. |
ARASH ARDAKANI et. al. | naacl | 2024-06-20 |
| 474 | CryptoGPT: A 7B Model Rivaling GPT-4 in The Task of Analyzing and Classifying Real-time Financial News Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: CryptoGPT: a 7B model competing with GPT-4 in a specific task — The Impact of Automatic Annotation and Strategic Fine-Tuning via QLoRAIn this article, we present a method aimed at refining a dedicated LLM of reasonable quality with limited resources in an industrial setting via CryptoGPT. |
Ying Zhang; Matthieu Petit Guillaume; Aurélien Krauth; Manel Labidi; | arxiv-cs.AI | 2024-06-20 |
| 475 | ChatGPT As Research Scientist: Probing GPT’s Capabilities As A Research Librarian, Research Ethicist, Data Generator and Data Predictor Summary Related Papers Related Patents Related Grants Related Venues Related Experts View Abstract: How good a research scientist is ChatGPT? We systematically probed the capabilities of GPT-3.5 and GPT-4 across four central components of the scientific process: as a Research … |
Steven A. Lehr; Aylin Caliskan; Suneragiri Liyanage; Mahzarin R. Banaji; | arxiv-cs.AI | 2024-06-20 |
| 476 | Generative AI for Enhancing Active Learning in Education: A Comparative Study of GPT-3.5 and GPT-4 in Crafting Customized Test Questions Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study investigates how LLMs, specifically GPT-3.5 and GPT-4, can develop tailored questions for Grade 9 math, aligning with active learning principles. |
Hamdireza Rouzegar; Masoud Makrehchi; | arxiv-cs.CL | 2024-06-19 |
| 477 | A Decision-Making GPT Model Augmented with Entropy Regularization for Autonomous Vehicles Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, the decision-making challenges associated with autonomous vehicles are conceptualized through the framework of the Constrained Markov Decision Process (CMDP) and approached as a sequence modeling problem. |
JIAQI LIU et. al. | arxiv-cs.RO | 2024-06-19 |
| 478 | Fine-Tuning BERTs for Definition Extraction from Mathematical Text Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we fine-tuned three pre-trained BERT models on the task of definition extraction from mathematical English written in LaTeX. |
Lucy Horowitz; Ryan Hathaway; | arxiv-cs.CL | 2024-06-19 |
| 479 | Generating Educational Materials with Different Levels of Readability Using LLMs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study introduces the leveled-text generation task, aiming to rewrite educational materials to specific readability levels while preserving meaning. |
Chieh-Yang Huang; Jing Wei; Ting-Hao ‘Kenneth’ Huang; | arxiv-cs.CL | 2024-06-18 |
| 480 | SwinStyleformer Is A Favorable Choice for Image Inversion Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper proposes the first pure Transformer structure inversion network called SwinStyleformer, which can compensate for the shortcomings of the CNNs inversion framework by handling long-range dependencies and learning the global structure of objects. |
Jiawei Mao; Guangyi Zhao; Xuesong Yin; Yuanqi Chang; | arxiv-cs.CV | 2024-06-18 |
| 481 | Adversarial Attacks on Multimodal Agents Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we show that multimodal agents raise new safety risks, even though attacking agents is more challenging than prior attacks due to limited access to and knowledge about the environment. |
Chen Henry Wu; Jing Yu Koh; Ruslan Salakhutdinov; Daniel Fried; Aditi Raghunathan; | arxiv-cs.LG | 2024-06-18 |
| 482 | What Makes Two Language Models Think Alike? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Do architectural differences significantly affect the way models represent and process language? We propose a new approach, based on metric-learning encoding models (MLEMs), as a first step to answer this question. |
Jeanne Salle; Louis Jalouzot; Nur Lan; Emmanuel Chemla; Yair Lakretz; | arxiv-cs.CL | 2024-06-18 |
| 483 | Vernacular? I Barely Know Her: Challenges with Style Control and Stereotyping Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We provide a thorough analysis and discussion of the results. |
ANKIT AICH et. al. | arxiv-cs.CL | 2024-06-18 |
| 484 | ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce ChatGLM, an evolving family of large language models that we have been developing over time. |
TEAM GLM et. al. | arxiv-cs.CL | 2024-06-18 |
| 485 | Promises, Outlooks and Challenges of Diffusion Language Modeling Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: For example, autoregressive token generation is notably slow and can be prone to \textit{exposure bias}. The diffusion-based language models were proposed as an alternative to autoregressive generation to address some of these limitations. |
Justin Deschenaux; Caglar Gulcehre; | arxiv-cs.CL | 2024-06-17 |
| 486 | Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Level Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we identify a pitfall of vanilla iterative DPO – improved response quality can lead to increased verbosity. |
JIE LIU et. al. | arxiv-cs.CL | 2024-06-17 |
| 487 | A Two-dimensional Zero-shot Dialogue State Tracking Evaluation Method Using GPT-4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose a two-dimensional zero-shot evaluation method for DST using GPT-4, which divides the evaluation into two dimensions: accuracy and completeness. |
Ming Gu; Yan Yang; | arxiv-cs.CL | 2024-06-17 |
| 488 | Cultural Conditioning or Placebo? On The Effectiveness of Socio-Demographic Prompting Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we systematically probe four LLMs (Llama 3, Mistral v0.2, GPT-3.5 Turbo and GPT-4) with prompts that are conditioned on culturally sensitive and non-sensitive cues, on datasets that are supposed to be culturally sensitive (EtiCor and CALI) or neutral (MMLU and ETHICS). |
SAGNIK MUKHERJEE et. al. | arxiv-cs.CL | 2024-06-17 |
| 489 | Minimal Self in Humanoid Robot Alter3 Driven By Large Language Model Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper introduces Alter3, a humanoid robot that demonstrates spontaneous motion generation through the integration of GPT-4, Large Language Model (LLM). |
Takahide Yoshida; Suzune Baba; Atsushi Masumori; Takashi Ikegami; | arxiv-cs.RO | 2024-06-17 |
| 490 | Large Language Model Tokenizer Bias: A Case Study and Solution on GPT-4o Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This misrepresentation results in the propagation of ‘under-trained’ or ‘untrained’ tokens, which perpetuate biases and pose serious concerns related to data security and ethical standards. We aim to dissect the tokenization mechanics of GPT-4o, illustrating how its simplified token-handling methods amplify these risks and offer strategic solutions to mitigate associated security and ethical issues. |
Jin Yang; Zhiqiang Wang; Yanbin Lin; Zunduo Zhao; | arxiv-cs.CL | 2024-06-17 |
| 491 | Look Further Ahead: Testing The Limits of GPT-4 in Path Planning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, they still face challenges with long-horizon planning. To study this, we propose path planning tasks as a platform to evaluate LLMs’ ability to navigate long trajectories under geometric constraints. |
Mohamed Aghzal; Erion Plaku; Ziyu Yao; | arxiv-cs.AI | 2024-06-17 |
| 492 | DB-GPT-Hub: Towards Open Benchmarking Text-to-SQL Empowered By Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present DB-GPT-Hub, an open benchmark suite for LLM-empowered text-to-SQL, which primarily focuses on tuning LLMs at large scales. |
FAN ZHOU et. al. | arxiv-cs.DB | 2024-06-17 |
| 493 | GPT-Powered Elicitation Interview Script Generator for Requirements Engineering Training Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We employ a prompt chaining approach to mitigate the output length constraint of GPT to be able to generate thorough and detailed interview scripts. |
Binnur Görer; Fatma Başak Aydemir; | arxiv-cs.SE | 2024-06-17 |
| 494 | WellDunn: On The Robustness and Explainability of Language Models and Large Language Models in Identifying Wellness Dimensions Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Language Models (LMs) are being proposed for mental health applications where the heightened risk of adverse outcomes means predictive performance may not be a sufficient litmus test of a model’s utility in clinical practice. |
SEYEDALI MOHAMMADI et. al. | arxiv-cs.AI | 2024-06-17 |
| 495 | Connecting The Dots: Evaluating Abstract Reasoning Capabilities of LLMs Using The New York Times Connections Word Game Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To deepen our understanding we create a taxonomy of the knowledge types required to successfully categorize words in the Connections game, revealing that LLMs struggle with associative, encyclopedic, and linguistic knowledge. |
PRISHA SAMADARSHI et. al. | arxiv-cs.CL | 2024-06-16 |
| 496 | Large Language Models for Automatic Milestone Detection in Group Discussions Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we investigate an LLM’s performance on recordings of a group oral communication task in which utterances are often truncated or not well-formed. |
ZHUOXU DUAN et. al. | arxiv-cs.CL | 2024-06-16 |
| 497 | Distilling Opinions at Scale: Incremental Opinion Summarization Using XL-OPSUMM Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To mitigate, we propose a scalable framework called Xl-OpSumm that generates summaries incrementally. |
SRI RAGHAVA MUDDU et. al. | arxiv-cs.CL | 2024-06-16 |
| 498 | Generating Tables from The Parametric Knowledge of Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: For evaluation, we introduce a novel benchmark, WikiTabGen which contains 100 curated Wikipedia tables. |
Yevgeni Berkovitch; Oren Glickman; Amit Somech; Tomer Wolfson; | arxiv-cs.CL | 2024-06-16 |
| 499 | ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Consequently, we present Video Diffusion GPT (ViD-GPT). |
Kaifeng Gao; Jiaxin Shi; Hanwang Zhang; Chunping Wang; Jun Xiao; | arxiv-cs.CV | 2024-06-16 |
| 500 | Breaking Boundaries: Investigating The Effects of Model Editing on Cross-linguistic Performance Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper strategically identifies the need for linguistic equity by examining several knowledge editing techniques in multilingual contexts. |
SOMNATH BANERJEE et. al. | arxiv-cs.CL | 2024-06-16 |
| 501 | KGPA: Robustness Evaluation for Large Language Models Via Cross-Domain Knowledge Graphs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper proposes a framework that systematically evaluates the robustness of LLMs under adversarial attack scenarios by leveraging knowledge graphs (KGs). |
AIHUA PEI et. al. | arxiv-cs.CL | 2024-06-16 |
| 502 | Exposing The Achilles’ Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce a novel dataset MWP-MISTAKE, incorporating MWPs with both correct and incorrect reasoning steps generated through rule-based methods and smaller language models. |
Joykirat Singh; Akshay Nambi; Vibhav Vineet; | arxiv-cs.CL | 2024-06-16 |
| 503 | ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning Via Shared Low-Rank Adaptation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study introduces an approach to optimize Parameter Efficient Fine Tuning (PEFT) for Pretrained Language Models (PLMs) by implementing a Shared Low Rank Adaptation (ShareLoRA). |
Yurun Song; Junchen Zhao; Ian G. Harris; Sangeetha Abdu Jyothi; | arxiv-cs.CL | 2024-06-15 |
| 504 | Multilingual Large Language Models and Curse of Multilinguality Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Multilingual Large Language Models (LLMs) have gained large popularity among Natural Language Processing (NLP) researchers and practitioners. These models, trained on huge datasets, show proficiency across various languages and demonstrate effectiveness in numerous downstream tasks. |
Daniil Gurgurov; Tanja Bäumel; Tatiana Anikina; | arxiv-cs.CL | 2024-06-15 |
| 505 | Optimizing Layer-Fused Scheduling of Transformer Networks on Multi-accelerator Platforms Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This work enables extensive hardware/mapping exploration by extending the DSE framework Stream towards support for transformers across a wide variety of hardware architectures and different execution schedules. |
Steven Colleman; Arne Symons; Victor J. B. Jung; Marian Verhelst; | arxiv-cs.AR | 2024-06-14 |
| 506 | GPT-4o: Visual Perception Performance of Multimodal Large Language Models in Piglet Activity Understanding Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The initial evaluation experiments in this study validate the potential of multimodal large language models in livestock scene video understanding and provide new directions and references for future research on animal behavior video understanding. |
Yiqi Wu; Xiaodan Hu; Ziming Fu; Siling Zhou; Jiangong Li; | arxiv-cs.CV | 2024-06-14 |
| 507 | The Devil Is in The Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we try to unveil the mystery of social bias inside language models by introducing the concept of {\sc Social Bias Neurons}. |
YAN LIU et. al. | arxiv-cs.CL | 2024-06-14 |
| 508 | Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Additionally bidirectional feature propagation is highly sensitive to inaccurate optical flow in blurry frames leading to error accumulation during the propagation process. To address these issues we propose BSSTNet Blur-aware Spatio-temporal Sparse Transformer Network. |
Huicong Zhang; Haozhe Xie; Hongxun Yao; | cvpr | 2024-06-13 |
| 509 | GPT-Fabric: Folding and Smoothing Fabric By Leveraging Pre-Trained Foundation Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose GPT-Fabric for the canonical tasks of fabric folding and smoothing, where GPT directly outputs an action informing a robot where to grasp and pull a fabric. |
Vedant Raval; Enyu Zhao; Hejia Zhang; Stefanos Nikolaidis; Daniel Seita; | arxiv-cs.RO | 2024-06-13 |
| 510 | Mean-Shift Feature Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Transformer models developed in NLP make a great impact on computer vision fields producing promising performance on various tasks. |
Takumi Kobayashi; | cvpr | 2024-06-13 |
| 511 | GROD: Enhancing Generalization of Transformer with Out-of-Distribution Detection Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper introduces a novel approach based on OOD detection, termed the Generate Rounded OOD Data (GROD) algorithm, which significantly bolsters the generalization performance of transformer networks across various tasks. |
Yijin Zhou; Yuguang Wang; | arxiv-cs.LG | 2024-06-13 |
| 512 | SDPose: Tokenized Pose Estimation Via Circulation-Guide Self-Distillation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Those transformer-based models that require fewer resources are prone to under-fitting due to their smaller scale and thus perform notably worse than their larger counterparts. Given this conundrum we introduce SDPose a new self-distillation method for improving the performance of small transformer-based models. |
SICHEN CHEN et. al. | cvpr | 2024-06-13 |
| 513 | OmniMotionGPT: Animal Motion Generation with Limited Data Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Our paper aims to generate diverse and realistic animal motion sequences from textual descriptions without a large-scale animal text-motion dataset. |
ZHANGSIHAO YANG et. al. | cvpr | 2024-06-13 |
| 514 | MoMask: Generative Masked Modeling of 3D Human Motions IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce MoMask a novel masked modeling framework for text-driven 3D human motion generation. |
Chuan Guo; Yuxuan Mu; Muhammad Gohar Javed; Sen Wang; Li Cheng; | cvpr | 2024-06-13 |
| 515 | MoST: Motion Style Transformer Between Diverse Action Contents Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This challenge lies in the lack of clear separation between content and style of a motion. To tackle this challenge we propose a novel motion style transformer that effectively disentangles style from content and generates a plausible motion with transferred style from a source motion. |
Boeun Kim; Jungho Kim; Hyung Jin Chang; Jin Young Choi; | cvpr | 2024-06-13 |
| 516 | ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictions IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Most existing studies are devoted to designing vision-specific transformers to solve the above problems which introduce additional pre-training costs. Therefore we present a plain pre-training-free and feature-enhanced ViT backbone with Convolutional Multi-scale feature interaction named ViT-CoMer which facilitates bidirectional interaction between CNN and transformer. |
Chunlong Xia; Xinliang Wang; Feng Lv; Xin Hao; Yifeng Shi; | cvpr | 2024-06-13 |
| 517 | Condition-Aware Neural Network for Controlled Image Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present Condition-Aware Neural Network (CAN) a new method for adding control to image generative models. |
Han Cai; Muyang Li; Qinsheng Zhang; Ming-Yu Liu; Song Han; | cvpr | 2024-06-13 |
| 518 | Visual Fact Checker: Enabling High-Fidelity Detailed Caption Generation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work we propose VisualFactChecker (VFC) a flexible training-free pipeline that generates high-fidelity and detailed captions for both 2D images and 3D objects. |
YUNHAO GE et. al. | cvpr | 2024-06-13 |
| 519 | Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present a large-scale study of linguistic bias exhibited by ChatGPT covering ten dialects of English (Standard American English, Standard British English, and eight widely spoken non-standard varieties from around the world). |
EVE FLEISIG et. al. | arxiv-cs.CL | 2024-06-13 |
| 520 | Permutation Equivariance of Transformers and Its Applications Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work we propose our definition of permutation equivariance a broader concept covering both inter- and intra- token permutation in the forward and backward propagation of neural networks. |
HENGYUAN XU et. al. | cvpr | 2024-06-13 |
| 521 | GPT-4V(ision) Is A Human-Aligned Evaluator for Text-to-3D Generation IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper presents an automatic versatile and human-aligned evaluation metric for text-to-3D generative models. |
TONG WU et. al. | cvpr | 2024-06-13 |
| 522 | General Point Model Pretraining with Autoencoding and Autoregressive Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Inspired by the General Language Model we propose a General Point Model (GPM) that seamlessly integrates autoencoding and autoregressive tasks in a point cloud transformer. |
ZHE LI et. al. | cvpr | 2024-06-13 |
| 523 | Complex Image-Generative Diffusion Transformer for Audio Denoising Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In order to enhance audio denoising performance, this paper introduces a complex image-generative diffusion transformer that captures more information from the complex Fourier domain. |
Junhui Li; Pu Wang; Jialu Li; Youshan Zhang; | arxiv-cs.SD | 2024-06-13 |
| 524 | AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, achieving faithful attributions for the entirety of a black-box transformer model and maintaining computational efficiency is an unsolved challenge. By extending the Layer-wise Relevance Propagation attribution method to handle attention layers, we address these challenges effectively. |
REDUAN ACHTIBAT et. al. | icml | 2024-06-12 |
| 525 | Accelerating Transformer Pre-training with 2:4 Sparsity Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: First, we define a “flip rate” to monitor the stability of a 2:4 training process. Utilizing this metric, we propose three techniques to preserve accuracy: to modify the sparse-refined straight-through estimator by applying the masked decay term on gradients, to determine a feasible decay factor in warm-up stage, and to enhance the model’s quality by a dense fine-tuning procedure near the end of pre-training. |
Yuezhou Hu; Kang Zhao; Weiyu Huang; Jianfei Chen; Jun Zhu; | icml | 2024-06-12 |
| 526 | Long Is More for Alignment: A Simple But Tough-to-Beat Baseline for Instruction Fine-Tuning IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: LIMA (NeurIPS 2023) and AlpaGasus (ICLR 2024) are state-of-the-art methods for selecting such high-quality examples, either via manual curation or using GPT-3.5-Turbo as a quality scorer. We show that the extremely simple baseline of selecting the 1,000 instructions with longest responses—that intuitively contain more learnable information and are harder to overfit—from standard datasets can consistently outperform these sophisticated methods according to GPT-4 and PaLM-2 as judges, while remaining competitive on the Open LLM benchmarks that test factual knowledge. |
Hao Zhao; Maksym Andriushchenko; Francesco Croce; Nicolas Flammarion; | icml | 2024-06-12 |
| 527 | GPT-4V(ision) Is A Generalist Web Agent, If Grounded IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we explore the potential of LMMs like GPT-4V as a generalist web agent that can follow natural language instructions to complete tasks on any given website. |
Boyuan Zheng; Boyu Gou; Jihyung Kil; Huan Sun; Yu Su; | icml | 2024-06-12 |
| 528 | Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we show that, for linearized transformer networks, ICL can be made explicit and permanent through the inclusion of bias terms. |
Brian K Chen; Tianyang Hu; Hui Jin; Hwee Kuan Lee; Kenji Kawaguchi; | icml | 2024-06-12 |
| 529 | Privacy-Preserving Embedding Via Look-up Table Evaluation with Fully Homomorphic Encryption Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Thus, our study proposes an efficient algorithm for privacy-preserving embedding via look-up table evaluation with HE(HELUT) by developing an encrypted indicator function (EIF) that assures high precision with the use of the approximate HE scheme(CKKS). |
Jae-yun Kim; Saerom Park; Joohee Lee; Jung Hee Cheon; | icml | 2024-06-12 |
| 530 | In-context Learning on Function Classes Unveiled for Transformers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Given some training examples, a pre-trained model can make accurate predictions on an unseen input. |
Zhijie Wang; Bo Jiang; Shuai Li; | icml | 2024-06-12 |
| 531 | Label-aware Hard Negative Sampling Strategies with Momentum Contrastive Learning for Implicit Hate Speech Detection Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose Label-aware Hard Negative sampling strategies (LAHN) that encourage the model to learn detailed features from hard negative samples, instead of naive negative samples in random batch, using momentum-integrated contrastive learning. |
Jaehoon Kim; Seungwan Jin; Sohyun Park; Someen Park; Kyungsik Han; | arxiv-cs.CL | 2024-06-12 |
| 532 | Asymmetry in Low-Rank Adapters of Foundation Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Inspired by an effort to investigate the different roles of LoRA matrices during fine-tuning, this paper characterizes and leverages unexpected asymmetry in the importance of low-rank adapter matrices. |
JIACHENG ZHU et. al. | icml | 2024-06-12 |
| 533 | An Empirical Study of Mamba-based Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In a controlled setting (e.g., same data), however, studies so far have only presented small scale experiments comparing SSMs to Transformers. To understand the strengths and weaknesses of these architectures at larger scales, we present a direct comparison between 8B-parameter Mamba, Mamba-2, and Transformer models trained on the same datasets of up to 3.5T tokens. |
ROGER WALEFFE et. al. | arxiv-cs.LG | 2024-06-12 |
| 534 | Stealing Part of A Production Language Model IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce the first model-stealing attack that extracts precise, nontrivial information from black-box production language models like OpenAI’s ChatGPT or Google’s PaLM-2. |
NICHOLAS CARLINI et. al. | icml | 2024-06-12 |
| 535 | Trainable Transformer in Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose a new efficient construction, Transformer in Transformer (in short, TINT), that allows a transformer to simulate and fine-tune more complex models during inference (e.g., pre-trained language models). |
Abhishek Panigrahi; Sadhika Malladi; Mengzhou Xia; Sanjeev Arora; | icml | 2024-06-12 |
| 536 | Algorithm and Hardness for Dynamic Attention Maintenance in Large Language Models IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Inspired by previous theoretical study of static version of the attention multiplication problem [Zandieh, Han, Daliri, and Karbasi ICML 2023, Alman and Song NeurIPS 2023], we formally define a dynamic version of attention matrix multiplication problem. |
Jan van den Brand; Zhao Song; Tianyi Zhou; | icml | 2024-06-12 |
| 537 | Adaptively Bypassing Vision Transformer Blocks for Efficient Visual Tracking Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, the slow speed of current trackers limits their applicability on devices with constrained computational resources. To address this challenge, we introduce ABTrack, an adaptive computation framework that adaptively bypassing transformer blocks for efficient visual tracking. |
XIANGYANG YANG et. al. | arxiv-cs.CV | 2024-06-12 |
| 538 | How Language Model Hallucinations Can Snowball IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: A major risk of using language models in practical applications is their tendency to hallucinate incorrect statements.To do this, we construct three question-answering datasets where LMs often state an incorrect answer which is followed by an explanation with at least one incorrect claim. |
Muru Zhang; Ofir Press; William Merrill; Alisa Liu; Noah A. Smith; | icml | 2024-06-12 |
| 539 | Timer: Generative Pre-trained Transformers Are Large Time Series Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To change the status quo of training scenario-specific small models from scratch, this paper aims at the early development of large time series models (LTSM). |
YONG LIU et. al. | icml | 2024-06-12 |
| 540 | Ditto: Quantization-aware Secure Inference of Transformers Upon MPC Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, the integration of quantization widely used in plaintext inference into the MPC domain remains unclear. To bridge this gap, we propose the framework named Ditto to enable more efficient quantization-aware secure Transformer inference. |
HAOQI WU et. al. | icml | 2024-06-12 |
| 541 | Enhancing Vision Transformer: Amplifying Non-Linearity in Feedforward Network Module Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we focus on improving the FFN module within the vision transformer. |
YIXING XU et. al. | icml | 2024-06-12 |
| 542 | Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERT Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To address these challenges, we first introduce LoCoV1, a 12 task benchmark constructed to measure long-context retrieval where chunking is not possible or not effective. We next present the M2-BERT retrieval encoder, an 80M parameter state-space encoder model built from the Monarch Mixer architecture, capable of scaling to documents up to 32K tokens long. |
Jon Saad-Falcon; Daniel Y Fu; Simran Arora; Neel Guha; Christopher Re; | icml | 2024-06-12 |
| 543 | Entropy-Reinforced Planning with Large Language Models for Drug Discovery Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Here we propose ERP, Entropy-Reinforced Planning for Transformer Decoding, which employs an entropy-reinforced planning algorithm to enhance the Transformer decoding process and strike a balance between exploitation and exploration. |
Xuefeng Liu; Chih-chan Tien; Peng Ding; Songhao Jiang; Rick L. Stevens; | icml | 2024-06-12 |
| 544 | PolySketchFormer: Fast Transformers Via Sketching Polynomial Kernels Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Recent theoretical results indicate the intractability of sub-quadratic softmax attention approximation under reasonable complexity assumptions. This paper addresses this challenge by first demonstrating that polynomial attention with high degree can effectively replace softmax without sacrificing model quality. |
Praneeth Kacham; Vahab Mirrokni; Peilin Zhong; | icml | 2024-06-12 |
| 545 | InstructRetro: Instruction Tuning Post Retrieval-Augmented Pretraining IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we introduce Retro 48B, the largest LLM pretrained with retrieval. |
BOXIN WANG et. al. | icml | 2024-06-12 |
| 546 | Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, future superhuman models will behave in complex ways too difficult for humans to reliably evaluate; humans will only be able to *weakly supervise* superhuman models. We study an analogy to this problem: can weak model supervision elicit the full capabilities of a much stronger model? |
COLLIN BURNS et. al. | icml | 2024-06-12 |
| 547 | Discrete Diffusion Modeling By Estimating The Ratios of The Data Distribution Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Crucially, standard diffusion models rely on the well-established theory of score matching, but efforts to generalize this to discrete structures have not yielded the same empirical gains. In this work, we bridge this gap by proposing score entropy, a novel loss that naturally extends score matching to discrete spaces, integrates seamlessly to build discrete diffusion models, and significantly boosts performance. |
Aaron Lou; Chenlin Meng; Stefano Ermon; | icml | 2024-06-12 |
| 548 | Do Efficient Transformers Really Save Computation? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we aim to understand the capabilities and limitations of efficient Transformers, specifically the Sparse Transformer and the Linear Transformer. |
KAI YANG et. al. | icml | 2024-06-12 |
| 549 | In-Context Principle Learning from Mistakes Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Nonetheless, all ICL-based approaches only learn from correct input-output pairs. In this paper, we revisit this paradigm, by learning more from the few given input-output examples. |
TIANJUN ZHANG et. al. | icml | 2024-06-12 |
| 550 | Fine-Tuned ‘Small’ LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classification Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we show that smaller, fine-tuned LLMs (still) consistently and significantly outperform larger, zero-shot prompted models in text classification. |
Martin Juan José Bucher; Marco Martini; | arxiv-cs.CL | 2024-06-12 |
| 551 | How Smooth Is Attention? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We provide a detailed study of the Lipschitz constant of self-attention in several practical scenarios, discussing the impact of the sequence length $n$ and layer normalization on the local Lipschitz constant of both unmasked and masked self-attention. |
Valérie Castin; Pierre Ablin; Gabriel Peyré; | icml | 2024-06-12 |
| 552 | Rethinking Generative Large Language Model Evaluation for Semantic Comprehension Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In response, we introduce an RWQ-Elo rating system, engaging 24 LLMs such as GPT-4, GPT-3.5, Google-Gemini-Pro and LLaMA-1/-2, in a two-player competitive format, with GPT-4 serving as the judge. |
Fangyun Wei; Xi Chen; Lin Luo; | icml | 2024-06-12 |
| 553 | Prodigy: An Expeditiously Adaptive Parameter-Free Learner IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose Prodigy, an algorithm that provably estimates the distance to the solution $D$, which is needed to set the learning rate optimally. |
Konstantin Mishchenko; Aaron Defazio; | icml | 2024-06-12 |
| 554 | What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We study the capabilities of the transformer architecture with varying depth. |
Xingwu Chen; Difan Zou; | icml | 2024-06-12 |
| 555 | Position: On The Possibilities of AI-Generated Text Detection Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce guidelines on the required text data quantity, either through sample size or sequence length, for reliable AI text detection, through derivations of sample complexity bounds. |
SOURADIP CHAKRABORTY et. al. | icml | 2024-06-12 |
| 556 | Prototypical Transformer As Unified Motion Learners Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we introduce the Prototypical Transformer (ProtoFormer), a general and unified framework that approaches various motion tasks from a prototype perspective. |
CHENG HAN et. al. | icml | 2024-06-12 |
| 557 | Outlier-Efficient Hopfield Layers for Large Transformer-Based Models IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce an Outlier-Efficient Modern Hopfield Model (termed `OutEffHop`) and use it to address the outlier inefficiency problem of training gigantic transformer-based models. |
JERRY YAO-CHIEH HU et. al. | icml | 2024-06-12 |
| 558 | Visual Transformer with Differentiable Channel Selection: An Information Bottleneck Inspired Approach Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose a novel and compact transformer block, Transformer with Differentiable Channel Selection, or DCS-Transformer. |
Yancheng Wang; Ping Li; Yingzhen Yang; | icml | 2024-06-12 |
| 559 | SpikeZIP-TF: Conversion Is All You Need for Transformer-based SNN Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we introduce a novel ANN-to-SNN conversion method called SpikeZIP-TF, where ANN and SNN are exactly equivalent, thus incurring no accuracy degradation. |
KANG YOU et. al. | icml | 2024-06-12 |
| 560 | Improving Autoformalization Using Type Checking Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Our analysis shows that the performance of these models is largely limited by their inability to generate formal statements that successfully type-check (i.e., are syntactically correct and consistent with types) – with a whopping 86.6% of GPT-4o errors starting from a type-check failure. In this work, we propose a method to fix this issue through decoding with type-check filtering, where we initially sample a diverse set of candidate formalizations for an informal statement, then use the Lean proof assistant to filter out candidates that do not type-check. |
Auguste Poiroux; Gail Weiss; Viktor Kunčak; Antoine Bosselut; | arxiv-cs.CL | 2024-06-11 |
| 561 | Anomaly Detection on Unstable Logs with GPT Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we report on an experimental comparison of a fine-tuned LLM and alternative models for anomaly detection on unstable logs. |
Fatemeh Hadadi; Qinghua Xu; Domenico Bianculli; Lionel Briand; | arxiv-cs.SE | 2024-06-11 |
| 562 | Towards Generalized Hydrological Forecasting Using Transformer Models for 120-Hour Streamflow Prediction Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Utilizing data from the preceding 72 hours, including precipitation, evapotranspiration, and discharge values, we developed a generalized model to predict future streamflow. |
Bekir Z. Demiray; Ibrahim Demir; | arxiv-cs.LG | 2024-06-11 |
| 563 | LT4SG@SMM4H24: Tweets Classification for Digital Epidemiology of Childhood Health Outcomes Using Pre-Trained Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents our approaches for the SMM4H24 Shared Task 5 on the binary classification of English tweets reporting children’s medical disorders. |
Dasun Athukoralage; Thushari Atapattu; Menasha Thilakaratne; Katrina Falkner; | arxiv-cs.CL | 2024-06-11 |
| 564 | Bilingual Sexism Classification: Fine-Tuned XLM-RoBERTa and GPT-3.5 Few-Shot Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This study aims to improve sexism identification in bilingual contexts (English and Spanish) by leveraging natural language processing models. |
AmirMohammad Azadi; Baktash Ansari; Sina Zamani; | arxiv-cs.CL | 2024-06-11 |
| 565 | Unveiling The Safety of GPT-4o: An Empirical Study Using Jailbreak Attacks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Specifically, this paper adopts a series of multi-modal and uni-modal jailbreak attacks on 4 commonly used benchmarks encompassing three modalities (ie, text, speech, and image), which involves the optimization of over 4,000 initial text queries and the analysis and statistical evaluation of nearly 8,000+ response on GPT-4o. |
Zonghao Ying; Aishan Liu; Xianglong Liu; Dacheng Tao; | arxiv-cs.CR | 2024-06-10 |
| 566 | LLM-dCache: Improving Tool-Augmented LLMs with GPT-Driven Localized Data Caching Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we introduce LLM-dCache to optimize data accesses by treating cache operations as callable API functions exposed to the tool-augmented agent. |
SIMRANJIT SINGH et. al. | arxiv-cs.DC | 2024-06-10 |
| 567 | Validating LLM-Generated Programs with Metamorphic Prompt Testing Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Research is required to comprehensively explore these critical concerns surrounding LLM-generated code. In this paper, we propose a novel solution called metamorphic prompt testing to address these challenges. |
Xiaoyin Wang; Dakai Zhu; | arxiv-cs.SE | 2024-06-10 |
| 568 | In-Context Learning and Fine-Tuning GPT for Argument Mining Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce an ICL strategy for ATC combining kNN-based examples selection and majority vote ensembling. |
Jérémie Cabessa; Hugo Hernault; Umer Mushtaq; | arxiv-cs.CL | 2024-06-10 |
| 569 | LLM-Powered Multimodal AI Conversations for Diabetes Prevention Summary Related Papers Related Patents Related Grants Related Venues Related Experts View Abstract: The global prevalence of diabetes remains high despite rising life expectancy with improved quality and access to healthcare services. The significant burden that diabetes imposes … |
Dung Dao; Jun Yi Claire Teo; Wenru Wang; Hoang D. Nguyen; | Proceedings of the 1st ACM Workshop on AI-Powered Q&A … | 2024-06-10 |
| 570 | Symmetric Dot-Product Attention for Efficient Training of BERT Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we propose an alternative compatibility function for the self-attention mechanism introduced by the Transformer architecture. |
Martin Courtois; Malte Ostendorff; Leonhard Hennig; Georg Rehm; | arxiv-cs.CL | 2024-06-10 |
| 571 | Annotation Alignment: Comparing LLM and Human Annotations of Conversational Safety Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: GPT-4 achieves a Pearson correlation of $r = 0.59$ with the average annotator rating, \textit{higher} than the median annotator’s correlation with the average ($r=0.51$). We show that larger datasets are needed to resolve whether LLMs exhibit disparities in how well they correlate with different demographic groups. |
Rajiv Movva; Pang Wei Koh; Emma Pierson; | arxiv-cs.CL | 2024-06-10 |
| 572 | Hidden Holes: Topological Aspects of Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The methods developed in this paper are novel in the field and based on mathematical apparatus that might be unfamiliar to the target audience. |
Stephen Fitz; Peter Romero; Jiyan Jonas Schneider; | arxiv-cs.CL | 2024-06-09 |
| 573 | Multi-attribute Auction-based Resource Allocation for Twins Migration in Vehicular Metaverses: A GPT-based DRL Approach Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, resource-intensive VTs updating and high mobility of vehicles require intensive computation, communication, and storage resources, especially for their migration among RSUs with limited coverages. To address these issues, we propose an attribute-aware auction-based mechanism to optimize resource allocation during VTs migration by considering both price and non-monetary attributes, e.g., location and reputation. |
YONGJU TONG et. al. | arxiv-cs.AI | 2024-06-08 |
| 574 | MaTableGPT: GPT-based Table Data Extractor from Materials Science Literature Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, the tables reported in materials science papers exist in highly diverse forms; thus, rule-based extractions are an ineffective approach. To overcome this challenge, we present MaTableGPT, which is a GPT-based table data extractor from the materials science literature. |
GYEONG HOON YI et. al. | arxiv-cs.CL | 2024-06-08 |
| 575 | G-Transformer: Counterfactual Outcome Prediction Under Dynamic and Time-varying Treatment Regimes Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we present G-Transformer for counterfactual outcome prediction under dynamic and time-varying treatment strategies. |
Hong Xiong; Feng Wu; Leon Deng; Megan Su; Li-wei H Lehman; | arxiv-cs.LG | 2024-06-08 |
| 576 | Automata Extraction from Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose an automata extraction algorithm specifically designed for Transformer models. |
Yihao Zhang; Zeming Wei; Meng Sun; | arxiv-cs.LG | 2024-06-08 |
| 577 | Do LLMs Recognize Me, When I Is Not Me: Assessment of LLMs Understanding of Turkish Indexical Pronouns in Indexical Shift Contexts Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present the first study examining indexical shift in any language, releasing a Turkish dataset specifically designed for this purpose. |
Metehan Oğuz; Yusuf Umut Ciftci; Yavuz Faruk Bakman; | arxiv-cs.CL | 2024-06-08 |
| 578 | VTrans: Accelerating Transformer Compression with Variational Information Bottleneck Based Pruning Related Papers Related Patents Related Grants Related Venues |