It is customary to go on Rosh Hashanah to a river to recite Tashlich. The following are the details and background of this custom.
A. Background-Its Start and Initiation:
This custom of Tashlich is first cited in the Sefer Maharil [14the century] in the section of Hilchos Rosh Hashanah and the Sefer Haminhagim of Tirna [who lived in that same generation], and is later recorded in numerous Sifrei Minhagim and Poskim of Ashkenazi Jewry. The custom is not recorded in the Talmud, or earlier Rishonim, such as the Rif and Rambam and Rosh, and is likewise omitted from the rulings of the Michaber in his Shulchan Aruch and surprisingly is also omitted from his encyclopedic commentary Beis Yosef, which was pended on the Tur and compiles teachings from all of the previous Poskim. From all this we can deduce that the custom became propagated sometime in the 13th or early 14th century, which was the time that the Maharil lived. Likewise, one can conclude that this custom originated amongst the communities of Ashkenazi Jewry, and was not followed amongst Sephardic communities until much later on, hence explaining its omission by the Michaber. In fact, when this custom is recorded in the writings of the Arizal, he writes it as an Ashkenazi custom. In his words, “The reason behind the custom that the Ashkenazim are accustomed to go on the first day…” Nonetheless, as is evident from the writings of the Sephardi Poskim, this tradition became adapted likewise amongst Sephardic community and as can be seen throughout the world today. However, this custom is not followed by all sects of world Jewry, as there are Ashkenazi communities even today who do not perform Tashlich and negate its significance or necessity. So was the ruling of the Gaon of Vilna, and so was testified by the Aruch Hashulchan to have been the custom of many Ashkenazi communities in his time to not perform Tashlich.
 See Admur 583:7; Siddur Admur; Rama 583:2 in name of Minhagim; Darkei Moshe 583:2; Toras Haolah of Rama 3:56; Drashos Maharil R”H; Sefer Haminhagim Tirana Rosh Hashanah p. 100; Mateh Moshe 820; Shlah Miseches R”H 214b; M”A 583:5; Shaar Hakavanos 90b; Peri Eitz Chaim 26:5; Mishnes Chassidim 9:14; Shvus Yaakov 3:42; Birkeiy Yosef 583:6; Machazik Bracha 583:4; Yosef Ometz 17:3; Ruach Chaim 583:6; Zechor Leavraham 583; Mateh Efraim 583:4; Shaareiy Teshuvah 583:6; Kitzur SHU”A 129:21; Ben Ish Chaiy Netzavim 12; Torah Leshma 145; Kaf Hachaim 583:30-31; M”B 583:8; Kaf Hachaim 583:30; Yabia Omer 4:47; Piskeiy Teshuvos 583:9; Otzer Minhagei Chabad 346;
Wording of Admur in 583:7: It is customary to go [on Rosh Hashanah] to a river which contains live fish as an omen that the evil eye should not have authority over us. It is also an omen for us to be blessed to multiply like fish. At the river one recites the verse “Yashuv Uyirachameinu” and “Vesashlich” etc. [as printed in the Siddur] as well as the verse of “Mi Kel Kamocha”. It is best to go [to recite Tashlich near a river which contains live fish] that is outside of the city. One is to go to recite Tashlich on the first day of Rosh Hashanah, after Mincha, prior to sunset and one is to recite there the verse of “Mi Kel Kamocha”.
Wording of Admur in Siddur: After Mincha on the first day of Rosh Hashanah, prior to sunset, it is proper to go to outside of the city to a well of water or a spring, as water represents Chesed while fish represent the level of “Eina Pikcha”. While there one is to recite the thirteen Middos that are in the verse “Mi Kel Kamocha” which correspond to the thirteen attributes of “Hashem Hashem Keil Rachum etc” One is to intend by each attribute to one of the attributes of the verse “Hashem Hashem Kiel Rachum etc”. However, they should not be verbalized. Afterwards one recites Min Hameitzar etc which correspond to the nine Middos of Hashem Erech Apayim etc”.
 See all Poskim in previous footnote who cite this custom
 Shaar Hakavanos 90b
 See Birkeiy Yosef 583:6; Machazik Bracha 583:4; Yosef Ometz 17:3; Ruach Chaim 583:6; Ben Ish Chaiy Netzavim 12; Torah Leshma 145; Kaf Hachaim 583:31 Kaf Hachaim 583:30; Yabia Omer 4:47
 In Maaseh Rav 209 it states that the Gr”a was not accustomed to go near a body of water to recite Tashlich; This is despite the fact that in Biur Hagr”a 583:11 he brings sources for the custom
 Aruch Hashulchan 583:4 records that many are accustomed not to perform Tashlich at all for reasons known to them and he concludes “and so is proper”.