The leaf damage pictured is classic "stippling" caused by lace bug feeding. These are a very common insect pest of azaleas, especially those stressed by receiving too much direct summer sun, reflected heat or reduced air circulation (such as from nearby walls, fences, or pavement), drought stress, and sometimes also from over-pruning that keeps the branches too crowded. They are easy to treat from the perspective of which insecticides work well for lace bug (horticultural oil or insecticidal soap, primarily, as both have low toxicity to other organisms), but hard to treat from the perspective of having to spray the leaf undersides to really get good control, and on a low-growing and dense plant, this is difficult to do well. (Thoroughly coating the leaf surfaces is important for the pesticide to work effectively.)

Branch dieback is probably due to moisture stress, either too much in the root zone at some point, or too little (drought), though periods of high heat are potentially also a factor. A likely cause is infection by Botryosphaeria canker, a fungus that takes advantage of plants under stress, especially from heat and drought. Root rot from over-watering, compacted soil, or poor soil drainage can also lead to branch dieback, and the two can be difficult to tell apart from symptoms alone. In either case, the only remedy is to do what you already have, which is to cut back all dead/dying growth since it will not recover. Does a roof downspout outlet or leaky gutter deposit extra water into this planting area? Were the plants monitored for watering needs during this year's drought (which has not yet abated)? Azaleas have comparatively shallow roots and have little tolerance for getting too dry, even though it takes a while for them to wilt due to their evergreen leaves.

You mention they are growing in partial sun...does the sun they do get directly occur in the morning, evening, or the hotter hours of the day (during summer)? If the latter, they might prefer a location with more midday shade.

You can learn about lace bug and other typical azalea pest, disease, or stress issues in our azalea diagnostic page. Leaves cannot heal from damage, even if pest suppression is successful, so it will take time for recovering plants to lose the bedraggled look of the lace bug feeding damage. Azaleas like these are evergreen, but like other evergreens, they do still shed old leaves, just not all at once the way deciduous plants do. After a few years, leaves become old enough to fall off, so eventually the cycle of new growth and old leaf shedding will cover-up and replace the current foliage that is showing heavy damage. Try to limit pruning to only removing what individual stems are growing "out of bounds" with regards to plant size or shape; don't shear the plants, as that promotes denser regrowth that can make pest outbreaks worse and harder to treat. Otherwise, azaleas don't really need routine pruning, and you can also probably skip fertilization also, since they should not need supplemental nutrients unless a laboratory soil test indicates a soil deficiency. (The test can also be a handy way to evaluate if the soil is acidic enough for azaleas to thrive, in case the brick and concrete nearby are leaching and raising the soil pH.)

If you decide instead to remove/replace the plants, neither the lace bugs nor any Botryosphaeria contamination will be present in the soil, so you won't have to worry about that, though either could always recolonize new plants if the replacement shrubs also become stressed. (Azalea lace bug only feeds on azaleas and rhododendrons, though, so if you planted an unrelated species, they will not be a problem.)

Miri