Cereal grains are botanically fruits, more specifically caryopses. "Caryopsis" is a generally unimportant term though, as "grain" is the more common and intuitive word for largely the same thing. They're the "seeds" from grasses.
While grains fall under the botanical definition of "fruit," they are clearly not fruits culinarily. While you may see corn pudding or rice pudding, those are desserts in spite of the grains, not because of them. You would never chow down on some raw barley for a sweet snack.
They also would fit the definition of vegetables. However, they are usually considered to be separate. Tthis is a reasonable separation to make. Nutritionally, cereal grains are usually much higher in carbohydrates, protein, and fiber but lower in vitamins and minerals. Grains are often ground up and made into bread or something, while vegetables are eaten in any way but that. Grains are dried, enabling them to keep for long periods of time, while vegetables either have to be eaten quickly or preserved by something like pickling.
One of the biggest different is how much people eat of each. Wheat, corn, and rice make up a humongous portion of the world's diet and have about as long as man has existed. "Vegetable" becomes kind of a pointless term if it includes everything you eat, so it would make sense to break the main boring staple of that out into its own category, especially since it is so different in other ways.
As a possible side-effect and other distinction, grains are always dried, either before or after picking. This is important for a staple food as it allowed it to be preserved to sustain people throughout the rest of the year. They are also commonly ground up and used to make bread.