Monitoring ephemeral gullies facilitates water planning and soil conservation. Artificial interpretation based on high spatial resolution images is the main method for monitoring ephemeral gullies in large areas; however, this method is time consuming. In this study, a semiautomatic method for extracting ephemeral gullies in loess hilly areas based on directional edge detection is proposed. First, the area where ephemeral gullies developed was extracted because the weak trace of ephemeral gullies in images can hardly be detected by most image detectors, which avoided the noise from other large gullies. Second, a Canny edge detector was employed to extract all edges in the image. Then, those edges along the direction where ephemeral gullies developed were searched and coded as candidate ephemeral gullies. Finally, the ephemeral gullies were identified through filtering of pseudo-gullies by setting the appropriate length threshold. Experiments in three loess hilly areas showed that accuracy ranged from 38.18% to 85.05%, completeness ranged from 82.35% to 92.86%, and quality ranged from 35.29% to 79.82%. The quality of the remote sensing images highly affected the results. The accuracy was significantly improved when the image was used with less grass and shrubs. The length threshold in directional searching also affected the accuracy. A small threshold resulted in additional noise and disconnected gullies, whereas a large threshold disregarded the short gullies. A reasonable threshold can be obtained through the index of quality. The threshold also exhibits a strong relationship with the average length of ephemeral gullies, and this relationship can help obtain the optimum threshold in the hilly area of the Northern Loess Plateau of China.