Daijizen No Mahoutsukai Ashuto, Sutareta Ryouchi De Slow Life summary:

Ashuto was the progeny of a great aristocratic family, but was chased out of the house when his magical aptitude was found to be aligned with plants and was labelled as a failure. At his father’s arrangement, he was to serve as the lord of the haunted forest, Obelstein. Somehow, he came to meet with a member of the rare high elves, Elumina and they lived together. Following that, countless rare species gathered in the forest, and a dragon of legends bestowed upon Ashuto with tremendous magic power—? Turned into a great mage overnight, Ashuto aims to create the greatest town with his plant magic!