ESG ratings have become increasingly important for capital market investors, firms, and regulators. They serve as key performance indicators of various socially responsible investment and management practices. We begin by showing that industry effects explain a significant portion of disagreement in ESG ratings. More importantly, by decomposing a firm’s ESG rating disagreement into industry- and firm-specific components, we find that the industry-level disagreement mainly arises from raters’ inconsistent classifications on firms’ industries. This practice is problematic since a firm is relatively evaluated based on different sets of peers depending on the rater. We further find that industry-level disagreement, rather than firm-specific disagreement, weakens the association between ESG ratings and the cross-section of stock returns. Our findings highlight that standard setters should pay close attention to highly diverse industry classifications and measurement regimes used by various ESG raters to mitigate significant disparities across ESG ratings.