The Talmud rules that one who omits Ya’aleh V’Yavo on the night of Rosh Chodesh is not obligated to repeat the amidah, because “we do not sanctify the month at night”. Based on this, R. Abbile Pasilover (cited in Chayyei Adam, klal 24:10) asserts that on the first night of Rosh Hashanah, one who forgets “HaMelekh HaKadosh” need not repeat the amidah.

Rav Soloveitchik (as cited by R. Herschel Schachter, Eretz haTzvi, 5:4-5) disagreed with this position, noting that if it were true, one could equally suffice with a complete weekday amidah on the night of Rosh HaShanah. (On this point, see R. Chaim Pinchas Sheinberg, Mishmeret Chaim, II, Hilkhot K’riat Shma U’Tefilah, 4, who distinguishes between the yom tov aspect of Rosh HaShanah, which is a function of the “reality” of Rosh Chodesh and begins at night, and the Rosh Chodesh aspect, which depends on the “kedushah” of Rosh Chodesh, which is only at daytime). Such a conclusion is clearly not the case, as the entire format of prayer would be wrong. This would be true as well of a prayer omitting “HaMelekh HaKadosh”. The Talmud’s statement is only in reference to Ya’aleh V’Yavo, which is an additional insertion into the amidah. The amidah is only repeated to allow its inclusion.

The explanation of the Talmud’s ruling would be as follows: when the moon was historically sanctified based on astronomical observation, the beit din established Rosh Chodesh. In modern days, the calendar is already calculated. It appears from the Rambam (Sefer HaMitzvot, 153) that the behavior of the Jewish people, in observance of Rosh Chodesh, is the establishing element. That behavior is expressed through the recitation of Ya’aleh V’Yavo. Thus, as the new moon is only sanctified during the day (which, as R. Schachter points out, is a distinct concept from the rule that beit din does not sit at night), that is the crucial act, and the amidah must be repeated to accommodate it. At night, when no sanctification takes place, there is no need to repeat the amidah, as it is not disqualified by the absence of Ya’aleh V’Yavo.

Tosafot (s.v. l’fi) suggest that the Talmud’s reasoning would only apply to the first night of Rosh Chodesh, and thus at the second night the amidah should be repeated. (See Ohr Sameach, Hil Tefilah 10:11). The conclusion of Tosafot is not to distinguish. In light of the above, this position is understood, and this is as well the conclusion of the Torat Rephael (II, 62) in a detailed responsum on the matter. (See also, at length, Resp. Shevet HaLevi, I, 131).

The Avnei Nezer (Responsa, O.C. 310:7) finds the Talmud’s ruling difficult in light of the fact that once the month is sanctified in the daytime, that status retroactively applies to the night. He suggests that since, at the nighttime, the sanctification has not actively begun yet, the status is not strong enough to command repition of the amidah. His student, the Kogaglover Rav (Resp. Eretz Tzvi, O.C. 26), suggests that in reality, the reason that the amidah need not repeated is similar to the reason that birkat hamazon is not repeated on Rosh Chodesh when Ya’aleh V’Yavo is omitted: the Rosh Chodesh meal is not obligatory, and the whole opportunity could have been willingly foregone. So too, ma’ariv can be prayed early, before Rosh Chodesh begins, and ya’aleh v’yavo would be unnecessary. However, if the new moon were sanctified at night, it would be improper to pray early and miss that obligation. However, since it is only sanctified in the daytime, and the night only retroactively afterward, there is no binding need to wait until dark for ma’ariv, and thus ya’aleh v’yavo is optional, and does not command repetition of the amidah.