To compare the effect and assess the consistence between Egger's test (ET) and Begg's test (BT), so as to choose the suitable method in publication bias (PB) diagnosis. We extracted PB-related information from 117 published meta-analysis papers through a thorough search in Medline database, and then compared the difference between ET and BT (while funnel plot [the most commonly used method to detect PB] as an important reference to judge the possible truth of PB) to detect the bias of the same data. Moreover, the correlation, agreement ratio (AR) and Kappa between ET and BT were also analyzed. We found that the rate of P-value less than 0.05 was 14.29% and 7.14% in ET and BT, respectively. Moreover, ET was commonly to get less P-value than BT (t=-2.22, P=0.03) to detect the bias of the same data. Though the results of ET and BT in most meta-analysis papers had a significant correlation (rs=0.68, P< 0.001), AR and Kappa were relatively low (AR=91.04%, Kappa=0.457, P=0.001). Moreover, ET and BT were not consistent to the PB diagnosis by the forest plot (AR=87.50% and 91.43%, respectively), and sometimes ET and BT were not consistent each other. From the results we conclude that ET is more sensitive in PB diagnosis than BT, but it is still not accurate. Thus, we recommend an associated application to improve the reliability of PB diagnosis.