First, let's look at that word "kindness", which is "chesed" in Hebrew. This word speaks of God's covenant loyalty to Israel, and thus to all of us who are in Christ Jesus! (Romans 8:15-17)
Becaused we have His Righteousness (2 Corinthians 5:21), we can rest assured that God will not stopping being gracious to us!
Check out that word "Depart", which is "mosh" in Hebrew.
Where does this word first appear in the Bible?
"He took not away the pillar of the cloud by day, nor the pillar of fire by night, from before the people." (Exodus 13:22)
"Took not away" is the same word.
Notice that it was the full Abrahamic Covenant of grace under which the Israelites were walking at this time.
This pillar is a picture of our Lord Jesus, whose shade protects us from the heat, and whose fire protects us from the cold:
"Behold, he that keepeth Israel shall neither slumber nor sleep.
"The LORD is thy keeper: the LORD is thy shade upon thy right hand.
"The sun shall not smite thee by day, nor the moon by night.
"The LORD shall preserve thee from all evil: he shall preserve thy soul. "(Psalm 121: 4-7)
We need to understand that because we are in Christ, because God has cut the New Covenant for us through His Son, we need never worry about His grace, His favor leaving us!
"10For this is the covenant that I will make with the house of Israel after those days, saith the Lord; I will put my laws into their mind, and write them in their hearts: and I will be to them a God, and they shall be to me a people:
11And they shall not teach every man his neighbour, and every man his brother, saying, Know the Lord: for all shall know me, from the least to the greatest.
12For I will be merciful to their unrighteousness, and their sins and their iniquities will I remember no more." (Hebrews 8: 10-12)
That's what kicks in the New Covenant, that His mercy endures forever, that the blood of Jesus cleanses us from all sin (1 John 1:9), and therefore He will remember our sins no more.
God's grace covers us, protects us, warms us, safeguards us, provides for us.
And we should rest assured that His grace, like that Pillar of Cloud and Fire, will never depart from us.