Let's talk about an aspect of game design that is very rarely noticed or understood by the audience.
Bloat is any content added to a game's rule book that arbitrarily increases its page count and doesn't actually add anything to play.
Padding is any content added to a game's rule book that arbitrarily increases the amount of time spent playing without actually adding anything to play.
Good and obvious examples of bloat are the fluff sections of a book; flavor text and art. However, it is also possible to create actual game rules content that either doesn't meaningfully do anything in play, or should not have been needed in the first place.
Good examples of Padding are XP systems for gaining levels, the gold and treasure distribution of certain games, the turn structure of combat, etc.
While both bloat and padding may both sound like bad things, given the description, the truth is that they're actually valuable and necessary elements of game design. Like any maligned element of the industry, the problem isn't the design element, but rather the product of bad designers.
For a closer look at bloat, let's look at another rarely understood game element in the D&D 5e design philosophy: ribbons. A ribbon is any racial or class trait that sounds like it does something in the game, but doesn't actually add anything to play in practice. In essence, ribbons are participation trophies that are used to make players think they're getting more out of a game mechanic than they actually are. A good example of a ribbon feature would be the druid's secret language, druidic. On paper, it works the same as any other language proficiency, but in practice it comes up so rarely that it effectively doesn't exist. Why is this? Because when adventure designers are making an adventure, they can't know what the party composition would be. As such, including an encounter where knowing druidic would be useful would isolate groups that don't include a druid. Then, even if a DM knows their party composition and makes an adventure that shines a light on the druid player via this mechanic, the result is just that the DM tells the druid what the coded information is, and then the druid repeats that information to the group... in which case, the DM might as well have just told the group and not bothered with the druidic element at all. Druid NPCs are so rare that the chances of a druid PC and NPC interacting is almost nil, and even if they do interact, they likely won't have any reason to speak in code. Likewise, the chances of a party containing more than one druid are fairly low, and again, it is unlikely that two PCs would need to speak in a coded language to each other. Thus, the druidic language is a redundant and non-functional mechanic.
Now, for a better understanding of padding, let's return to my earlier example of the XP system needed to gain levels. In D&D 5th edition, the developers did extensive market research to find out what levels of play were most enjoyable to the most players. When making the XP chart, they reduced the XP threshold for levels people don't like, such as first, and increased the XP threshold on levels people did like such as third and fifth. This allowed people to get out of the lame levels faster, and also forced them to game for longer in the cool levels where most of the fun is available to players. Thus, the XP thresholds are a rule that don't change the game's play at the table, but rather the temporal experience of play. D&D is full of padding, especially in the combat system, that serves only to force people to spend more time in aspects of the game that are understood to be most important or fun.
Bloat, done badly, can be utterly infuriating however. Let's take Star Trek Adventures for example. The core rule book is about twice as thick as it actually needs to be. This is mostly due to large sections of the book being just fluff; mostly flavor text. The problem with that is they mix rules in with the fluff randomly in a way that doesn't clearly distinguish the rules from the fluff. This makes reading the book a chore, and referencing the rules an infuriating pain in the butt. The rulebook also has a lot of its content duplicated in the form of copy-pasta sections in its second half. All of this makes a book that is hefty and user-unfriendly.
Padding, done badly, extends play without actually making that play enjoyable. For example, hit points in D&D 5e. As creatures gain levels or CR, their HP totals increase dramatically, but damage dice output doesn't keep up with this growth. The end result is that as the players gain levels they can fight increasingly powerful monsters which take exponentially longer to kill because of their arbitrarily large hit points. The PCs, too, are padded with an excess of hit points so character death is uncommon enough that even that won't normally speed up combat resolution. Now, on some monsters that are fun to fight, this is a good thing. However, monsters that are just giant HP lumps that walk around and hit stuff are crazy boring and are often padded way more than should be necessary. Ogres are a good example of this. The result is a sizeable number of players and DMs who complain about campaign sessions that last for hours consisting of nothing but a single boring fight that just drags on and on.
So now that we understand what bloat and padding are, let's talk about why they're important, aside from the explanations already given. In a word, money. It's always about money. The main reason games contain bloat and padding is money. See, professional game designers need to convince people to buy and get hooked on playing their games. They also need to turn a profit on sales. I'll let you in on a marketing secret: The better something seems and the lower its price, the more likely we are to assume there's something wrong with it and not buy it. Thus, ironically, in order to make an RPG more accessible to a larger audience, you actually need to sell it for a price that is slightly higher than the product would seem to be worth on the shelf! However, people are unlikely to spend that top dollar if they don't feel like they're getting something tangible that justifies that cost. As such, developers need to find ways to arbitrarily fatten their books with material people will find interesting without overbuilding the game's actual system or content the way BattleTech did. The result is that people who buy the books are less likely to feel buyer's remorse after purchasing, because the book is physically nice enough and heavy enough that the price seems justified. This is where padding comes in; by forcing people to spend more time on the fun parts of the game and less time on the unfun parts of the game, the developers increase the likely hood of a player enjoying themselves and wanting to play more. The more hours, days, weeks, months, years, etc the player gets padded into playing, the more supplemental content can be marketed and sold to that player, which in turn adds more enjoyable content, and perpetuates the cycle.
I, personally, avoid using bloat and padding in my designs unless they add to the actual gameplay experience. I feel like the profit-motivated aspects of game design are cynical and manipulative, and I just don't like that fact. This attitude has resulted in my single-page-engine philosophy. That is to say, if the full rules of your game system can't fit on a single double sided sheet of paper, your game is either too complex or very inefficiently written. I proved my philosophy to myself a long time ago when I made a simplified D&D engine that stripped away the content from the system and focused on crunch alone. I have followed that design philosophy ever since. The upshot of this, is that my games are not particularly marketable. No sane person would spend 60$ on a single sheet of paper containing the full rules to a game system, no matter how expansive the play experience may be. So I can't really sell my designs. Since I give my content away for free, people are likely to assume it's bad (like a free couch that smells like cat piss) so it's hard to promote to new players. I am OK with this, because I mostly just do this as a hobby, and I feel like the only reason people can do this professionally is the symptom of frankly unethical marketing ploys.
