You don't need a degree, but you should get a DBA (doing business as) certificate (also known as an assumed name certificate). You can get that from the county offices. And you will need the DBA certificate to open a business bank account. You will also need the business bank acct to set yourself up to accept credit cards (should you opt for that).

What you should look into is whether or not you can legally carry out your bakery/delivery business out of your home. Some cities don't allow it. Some may require a license or may need to pay a visit to see if your space is compliant with regulations that govern the making and selling of food items from a private residence.

Again, your county and city offices should be able to guide you.