publications

For an updated list please visit Google Scholar. * denotes equal contribution.

2024

1. Preprint

   

   FishVista: A Multi-Purpose Dataset for Understanding and Identification of Visual Traits from Images

   Kazi Sajeed Mehrab, M. Maruf, Arka Daw, H.B. Manogaran, A. Neog, and 12 more authors

   In Review, 2024

   Abs arXiv Code

   The availability of large datasets of organism images combined with advances in artificial intelligence (AI) has significantly enhanced the study of organisms through images, unveiling biodiversity patterns and macro-evolutionary trends. However, existing machine learning (ML)-ready organism datasets have several limitations. First, these datasets often focus on species classification only, overlooking tasks involving visual traits of organisms. Second, they lack detailed visual trait annotations, like pixel-level segmentation, that are crucial for in-depth biological studies. Third, these datasets predominantly feature organisms in their natural habitats, posing challenges for aquatic species like fish, where underwater images often suffer from poor visual clarity, obscuring critical biological traits. This gap hampers the study of aquatic biodiversity patterns which is necessary for the assessment of climate change impacts, and evolutionary research on aquatic species morphology. To address this, we introduce the Fish-Visual Trait Analysis (Fish-Vista) dataset—a large, annotated collection of about 60K fish images spanning 1900 different species, supporting several challenging and biologically relevant tasks including species classification, trait identification, and trait segmentation. These images have been curated through a sophisticated data processing pipeline applied to a cumulative set of images obtained from various museum collections. Fish-Vista ensures that visual traits of images are clearly visible, and provides fine-grained labels of various visual traits present in each image. It also offers pixel-level annotations of 9 different traits for 2427 fish images, facilitating additional trait segmentation and localization tasks. The ultimate goal of Fish-Vista is to provide a clean, carefully curated, high-resolution dataset that can serve as a foundation for accelerating biological discoveries using advances in AI. Finally, we provide a comprehensive analysis of state-of-the-art deep learning techniques on Fish-Vista.

2. NeurIPS 24

   

   VLM4Bio: A Benchmark Dataset to Evaluate Pretrained Vision-Language Models for Trait Discovery from Biological Images

   M. Maruf^*, Arka Daw^*, Kazi Sajeed Mehrab, H.B. Manogaran, A. Neog, and 17 more authors

   In NeurIPS, 2024

   Abs arXiv Code

   Images are increasingly becoming the currency for documenting biodiversity on the planet, providing novel opportunities for accelerating scientific discoveries in the field of organismal biology, especially with the advent of large vision-language models (VLMs). We ask if pre-trained VLMs can aid scientists in answering a range of biologically relevant questions without any additional fine-tuning. In this paper, we evaluate the effectiveness of 12 state-of-the-art (SOTA) VLMs in the field of organismal biology using a novel dataset, VLM4Bio, consisting of 469K question-answer pairs involving 30K images from three groups of organisms: fishes, birds, and butterflies, covering five biologically relevant tasks. We also explore the effects of applying prompting techniques and tests for reasoning hallucination on the performance of VLMs, shedding new light on the capabilities of current SOTA VLMs in answering biologically relevant questions using images.

2023

1. AAAI 23 Workshop

   Phylo-GNN: Phylogeny-guided Graph Neural Network Approach for Fine-Grained Image Trait Identification

   Kazi Sajeed Mehrab, Arka Daw, M. Maruf, and Anuj Karpatne

   In First Imageomics Workshop at AAAI, 2023
2. Let There Be Order: Rethinking Ordering in Autoregressive Graph Generation

   Jie Bu, Kazi Sajeed Mehrab, and Anuj Karpatne

   2023

2021

1. ACL 2021 Workshop

   Text2app: A framework for creating android apps from text descriptions

   Masum Hasan^*, Kazi Sajeed Mehrab^*, Wasi Uddin Ahmad, and Rifat Shahriyar

   NLP for Programming Workshop at ACL, 2021
2. ACL 21

   Codesc: A large code-description parallel dataset

   Masum Hasan, Tanveer Muttaqueen, Abdullah Al Ishtiaq, Kazi Sajeed Mehrab, Md Mahim Anjum Haque, and 4 more authors

   Finding of the Association of Computational Linguistics, ACL 2021, 2021
3. NLP

   Bert2code: Can pretrained language models be leveraged for code search?

   Abdullah Al Ishtiaq, Masum Hasan, Md Mahim Anjum Haque, Kazi Sajeed Mehrab, Tanveer Muttaqueen, and 3 more authors

   2021