There are two kinds of melee attacks in Phoenix Point, a “bash” with any item and “strike” with a dedicated melee weapon.
*Note that both skills benefit from the Close Quarters Specialist third-row skill (Proficiency and 20% damage buff with shotguns and melee weapons)
Striking with a dedicated melee weapon requires to be on an adjacent tile to the target and to be proficient in melee weapons [can anyone confirm that otherwise there is a chance to fumble?]. All melee weapons cost 2APs to strike with (though in the Blood and Titanium DLC there is a bionic augmentation that allows reducing cost of melee weapons by 1AP, to a minimum of 1AP).
Melee weapons can also be used to bash (which costs only 1AP) but for much less damage.
There is no free aim, or body part targeting with melee attacks, which means that some RNG is involved because any body part can receive the hit and different body parts have different armor.
The big advantages of melee weapons are the high damage per strike, which guarantees easily overcoming the toughest armors, that they have some secondary useful property (like armor penetration, or inflicting paralysis, bleeding, or shock damage) and that they are all one-handed weapons.
Best melee weapon
Though there are different melee weapons, in practice there is little (if any) reason to use anything other than Marduk’s Fist (a hammer), which does 160 base damage per strike and 200 shock. Note that because of RNG, often whether the target will be dazed or not will depend on the body part that was hit.
*Note that the Neuralizer doesn’t require melee proficiency and costs 1AP to use, but for all other intents and purposes acts like a dedicated melee weapon. Most importantly, to inflict paralysis damage it has to be used to Strike an enemy, not Bash it.
Bashing with an item an adjacent enemy is always available, provided that the soldier’s arms have not been disabled (for items requiring 2 arms to operate, both arms are needed to bash with the item) and always costs 1 AP.
The damage of the attack is calculated by multiplying the weight of the item used for bashing by the strength of the character doing it. So a soldier with 15 Strength wielding an Assault Rifle that weighs 3 units will do 3*15=45 damage with the bash.
The item that is used for bashing can also suffer damage [if anyone can shed a light on how the damage to the weapon is calculated, it would be much appreciated; in my experience, it seems to be more likely and higher as the damage to the target from the bash is higher], so brittle weapons like Sniper Rifles make for poor candidates.
(Weapons that get damaged in combat will be automatically repaired after the battle, but not so the weapons that were destroyed).
Because bash scales with strength, it can actually do as much as damage as Marduk’s Fist but for half the APs. A character with 32 Strength (that can only be achieved with max Strength and the Resourceful third-row skill) wielding a heavy weapon (e.g. Deceptor, Hel Cannon, Grenade Launcher - all of which weigh 5) will do 32 * 5 = 160. This is a 100% increase in damage per AP, but really anything above 80 damage (that is, with Strength of 17 or more) would seem to make bash preferable to melee.
There are a few caveats, however:
Melee weapons are one-handed while bashing with a heavy weapon requires both hands. This doesn’t seem very important… Until your heavy has one arm disabled.
Bashing can damage the weapon, and at very high Strength the damage seems to be significantly higher (I have seen up to 42 damage to the weapon from a single bash from a character with 32 Strength), which means that with very high Strength you might get no more than 4 bashes in before the weapon breaks (for example, Hel Cannon and Deceptor have 150 HPs).
Even though the damage per AP is important, when it comes to melee weapons remember that their big advantage over other weapons is the high damage per strike. A basic AR does more damage than a Marduk’s Fist (180 vs 160) for the same 2 APs (and over distance, too!). However, AR fires in burst and each shot does only 30 damage, which means that whereas against a target with 50 armor, the AR will do 0 damage, a Marduk’s Fist will do 110 damage. Similarly, even though 2 bashes of 90 damage each add to 180 pts of damage, against a target with 50 armor they would only do 80 in total.
For bash to compete with melee in damage, a serious investment in Strength is required. (This is not too big of an issue currently, as with the latest change to character progression you will have more than enough SPs to completely max out the stats of your veterans by late game.)
If you have the DLC, you can augment your soldiers with the melee bionic torso, which reduces the cost of melee attacks by 1AP.
Conclusion: The big advantage of bash is that it costs 1 AP and that with high enough Strength it can do as much as damage as a dedicated melee attack that costs 2 AP. This makes it ideal for Heavies (who have the Brawler skill - LVL2 Passive Heavy Skill, adds 50% damage to bash and melee attacks), that can jetpack to an enemy and club it (which is the best thing they can usually do after jetpacking anyway). It is also very useful for other builds, to take care of worms and mindfraggers. However, a dedicated melee fighter can be devastating, especially when considering all the buffs available to melee and that they don’t need to care about accuracy.