Absolutely! Go into “Appearance > Menus”, and open your menu items.
When they’re opened, you’ll see they each have a “Navigation label”.
In the navigation label before the text, add the HTML for your font awesome icon.
For example, the home icon:
<i class="fa fa-home"></i>
Let me know if you need more info 🙂