Soybean, the most popular golden bean of America, is widely known for its fat free food products. Richness in Protein makes it one of the best suggested meals which can be consumed in the form of pulses, oil, food for animals etc. The quality of such products is mainly dependent on the quality of Soybean procured as fresh farm produce. Governments and regional authorities have already defined the standards for quality assessment and grading of Soybean, which are meant to be followed in the commercial market while trading. Presently, visual inspection is the preferred way to conduct physical quality assessment of Soybean and it is performed by an expert person, at the time of procurement of Soybean based on the standards recommended by the buying authority. Physical parameters of Soybean kernel like color, growth corresponding to size, damage, fungi/ disease as well as mixing of other material/ objects in soybean sample influence the grade of that sample during quality assessment. Dependency of this process on human expert usually harms the accuracy in assessment. Therefore, an automated machine is desired to be a suitable solution to address this issue and can benefit in terms of increased accuracy, reliability, and reduced response time. Worldwide researchers are working on designing such automated systems for different type of fruits, grains etc. However, in case of Soybean, up till now we are successful in cleaning, sorting, color detection, and also, through various image processing techniques and algorithms researchers could detect the anomalies present in grain sample. But an automated system for the quality assessment and grading of Soybean according to an International Standard is yet to be implemented. In this paper, we propose a two-stage model for the quality assessment of Soybean; first stage focuses on image processing techniques like Image acquisition, preprocessing and feature extraction Likewise the second stage works on classification of Soybean kernels and sample grading with the help of a machine learning technique on the basis of an International Standard.