Is there Gender Bias in Dependency Parsing? Revisiting ``Women's Syntactic Resilience''
P. Go, and A. Falenska. Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP), page 269--279. Bangkok, Thailand, Association for Computational Linguistics, (August 2024)
Abstract
In this paper, we revisit the seminal work of Garimella et al. 2019, who reported that dependency parsers learn demographically-related signals from their training data and perform differently on sentences authored by people of different genders. We re-run all the parsing experiments from Garimella et al. 2019 and find that their results are not reproducible. Additionally, the original patterns suggesting the presence of gender biases fail to generalize to other treebank and parsing architecture. Instead, our data analysis uncovers methodological shortcomings in the initial study that artificially introduced differences into female and male datasets during preprocessing. These disparities potentially compromised the validity of the original conclusions.
%0 Conference Paper
%1 go-falenska-2024-gender
%A Go, Paul
%A Falenska, Agnieszka
%B Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP)
%C Bangkok, Thailand
%D 2024
%E Faleńska, Agnieszka
%E Basta, Christine
%E Costa jussà, Marta
%E Goldfarb-Tarrant, Seraphina
%E Nozza, Debora
%I Association for Computational Linguistics
%K iris iris3d
%P 269--279
%T Is there Gender Bias in Dependency Parsing? Revisiting ``Women's Syntactic Resilience''
%U https://aclanthology.org/2024.gebnlp-1.17
%X In this paper, we revisit the seminal work of Garimella et al. 2019, who reported that dependency parsers learn demographically-related signals from their training data and perform differently on sentences authored by people of different genders. We re-run all the parsing experiments from Garimella et al. 2019 and find that their results are not reproducible. Additionally, the original patterns suggesting the presence of gender biases fail to generalize to other treebank and parsing architecture. Instead, our data analysis uncovers methodological shortcomings in the initial study that artificially introduced differences into female and male datasets during preprocessing. These disparities potentially compromised the validity of the original conclusions.
@inproceedings{go-falenska-2024-gender,
abstract = {In this paper, we revisit the seminal work of Garimella et al. 2019, who reported that dependency parsers learn demographically-related signals from their training data and perform differently on sentences authored by people of different genders. We re-run all the parsing experiments from Garimella et al. 2019 and find that their results are not reproducible. Additionally, the original patterns suggesting the presence of gender biases fail to generalize to other treebank and parsing architecture. Instead, our data analysis uncovers methodological shortcomings in the initial study that artificially introduced differences into female and male datasets during preprocessing. These disparities potentially compromised the validity of the original conclusions.},
added-at = {2024-08-28T11:19:05.000+0200},
address = {Bangkok, Thailand},
author = {Go, Paul and Falenska, Agnieszka},
biburl = {https://puma.ub.uni-stuttgart.de/bibtex/264b6bb9171cb126ee4e0b88a307a4e19/iris},
booktitle = {Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP)},
editor = {Fale{\'n}ska, Agnieszka and Basta, Christine and Costa juss{\`a}, Marta and Goldfarb-Tarrant, Seraphina and Nozza, Debora},
interhash = {3d8ac9dbebbb1eb1ff46972a1c459c89},
intrahash = {64b6bb9171cb126ee4e0b88a307a4e19},
keywords = {iris iris3d},
month = aug,
pages = {269--279},
publisher = {Association for Computational Linguistics},
timestamp = {2024-10-02T23:14:48.000+0200},
title = {Is there Gender Bias in Dependency Parsing? Revisiting {``}Women{'}s Syntactic Resilience{''}},
url = {https://aclanthology.org/2024.gebnlp-1.17},
year = 2024
}
