Agricultural field management can directly affect biodiversity in rural areas. Pesticide reduction has been hypothesized to increase bird and bat activity in the field because food sources such as weeds and insects are more abundant in comparison to conventional farm management. Additionally, structural elements such as hedgerows in the surrounding landscape can serve as nesting habitats and flight corridors. Audio sensors can be easily installed at various locations and provide a largely autonomous, non-invasive technique for biodiversity monitoring under field conditions. AI-driven audio analysis methods allow to detect characteristic sound events associated with particular animal species. In this paper, we explore state-of-the-art deep-learning based audio analysis methods such as convolutional neural networks and pre-trained audio embeddings for bird and bat activity detection and classification in audio and ultrasonic recordings. Our recordings were derived from AudioMoth devices installed in the framework of a designed, large-scale experiment comprising arable fields with different field management and landscape surroundings. The applied methods allow for a detailed analysis of long-term audio recordings in order to draw conclusions on the influence of different agricultural field management techniques on local biodiversity. We show that AI-based audio monitoring is a promising tool for non-invasive long-term large-scale biodiversity assessment.