Currently, mainstream AI alignment methods such as Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO) rely on high-quality human preference feedback data.