A booming population around the world raises the concern of shortages of food resources in this new era. Thus, monitoring and managing crop production is extremely essential, especially rice crops, as they are the fundamental food source for most countries. Several challenges need to be addressed in this case, such as the classification of farmland from various land usages, precise monitoring of rice seedlings, and segmentation of rice growth. By leveraging advanced technologies such as drone imagery and machine learning, this paper proposed a new integrated pipeline for rice field classification and growth monitoring: a combination of convolutional neural networks (CNNs), You Only Look Once (YOLO), and modified U-Net models. These models were used in stages, specifically for paddy field classification, rice seedling detection, and rice growth segmentation. Substantial measurements and analysis have been carried out to verify the performance of the proposed system, including an accuracy of at least 85%, low classification/segmentation loss below 0.35, and high detection recall above 0.9. Thus, the findings highlight how combining different machine learning models with aerial photography can revolutionise conventional farming methods for better efficacy.