• Amused
  • Angry
  • Annoyed
  • Awesome
  • Bemused
  • Cocky
  • Cool
  • Crazy
  • Crying
  • Depressed
  • Down
  • Drunk
  • Embarrased
  • Enraged
  • Friendly
  • Geeky
  • Godly
  • Happy
  • Hateful
  • Hungry
  • Innocent
  • Meh
  • Piratey
  • Poorly
  • Sad
  • Secret
  • Shy
  • Sneaky
  • Tired
  • Page 1 of 2 12 LastLast
    Results 1 to 20 of 28

    Thread: Great Blue Heron Detection - Artificial Intelligence

    1. #1
      matiasklein is offline Junior Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jun 2017
      Location
      Philadelphia, PA
      Posts
      8

      Great Blue Heron Detection - Artificial Intelligence

      A great blue heron has been eating my favorite koi. Even my ever vigilant watchdog can't consistently detect this bird since it a silent, stealth killer. The heron has also killed koi through my pond net.

      I had enough, so I created an artificial intelligence program to detect and deploy countermeasures so I can finally remove my pond net and relax.

      Check out this video of a great blue heron being detected in the middle of the night when I was sleeping. Let me know what you think!


    2. #2
      stevek's Avatar
      stevek is offline Supporting Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jan 2010
      Location
      wakefield, RI
      Posts
      1,101
      Unless I missed it, what is the countermeasure ? The camera picks up the bird, but what keeps the bird out of the pond ?

    3. #3
      matiasklein is offline Junior Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jun 2017
      Location
      Philadelphia, PA
      Posts
      8
      Quote Originally Posted by stevek View Post
      Unless I missed it, what is the countermeasure ? The camera picks up the bird, but what keeps the bird out of the pond ?
      Right now I'm triggering a speaker with a dog barking and kids screaming. Seems to work pretty well. However, countermeasures can be whatever you want: bull horn, drone, water cannon, nerf gun, etc.

    4. #4
      spkennyva is offline Senior Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jan 2012
      Location
      Suffolk, VA
      Posts
      106
      Awesome. I attempted the same thing about a year ago. My camera would FTP images to a Raspberry Pi inside. The Pi would poll the receiving directory and when a change was detected it would run the AI program. Of course at that point firing off countermeasures was easy. I was using Darknet and although Darknet would detect Heron's reliably from stock images, I never felt confident to deploy it since my Wi-Fi was sketchy at the camera. I was also hoping to that the system would discriminate between people and herons, so that set the bar higher, but you seem that have that part well covered.

      Having dealt with herons for many years, I believe that you'll likely need to ramp up your countermeasures beyond sound alone.

      Can you provide more details on how you have your system set up?

    5. #5
      matiasklein is offline Junior Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jun 2017
      Location
      Philadelphia, PA
      Posts
      8
      Quote Originally Posted by spkennyva View Post
      Awesome. I attempted the same thing about a year ago. My camera would FTP images to a Raspberry Pi inside. The Pi would poll the receiving directory and when a change was detected it would run the AI program. Of course at that point firing off countermeasures was easy. I was using Darknet and although Darknet would detect Heron's reliably from stock images, I never felt confident to deploy it since my Wi-Fi was sketchy at the camera. I was also hoping to that the system would discriminate between people and herons, so that set the bar higher, but you seem that have that part well covered.

      Having dealt with herons for many years, I believe that you'll likely need to ramp up your countermeasures beyond sound alone.

      Can you provide more details on how you have your system set up?
      Thanks! Agree with you on more aggressive countermeasures. I suspect bird will get used to sound and eventually ignore it. I'll probably end up flying a small drone with blinking lights around the pond whenever the heron is detected, but for now I'm starting with less complex countermeasures like sound. Would welcome thoughts on what folks think would work best.

      As for my setup, I have a WiFi camera streaming video via RTSP to AI server (laptop) where I am doing real-time processing of the video. When a heron is detected, the AI server calls an API on my smart speaker (also connected via WiFi). I'm using a custom neural network that was specifically trained on a heron dataset, so am not seeing false alarms on people or other animals.

    6. #6
      BeyondKoi's Avatar
      BeyondKoi is offline Supporting Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      May 2010
      Location
      Fairfax, VA
      Posts
      304
      Quote Originally Posted by matiasklein View Post
      Right now I'm triggering a speaker with a dog barking and kids screaming. Seems to work pretty well. However, countermeasures can be whatever you want: bull horn, drone, water cannon, nerf gun, etc.
      Great Idea to use video with objects detection. I actually did a similar thing but using smart home sensors and smart speaker instead. I installed a roof sensor (where the heron has always landed before going to the pond. That sensor would activate the main sensor and send me a text message to alert me that "the heron is coming" . Then when the heron reaches the pond, the main sensor will trigger the speaker with dog barking sound. However, IR sensors caused too many false alarms. For countermeasures, triggering a speaker with dogs barking worked well for me. The heron was terrified with the dog barking sound and flew for his life and almost hit the tree (see video) I was also thinking of building (or buying and modifying) an automation Halloween prop if the dog sound didn't work, or simply using a inflatable tube man and only activate the fan when heron is detected. But so far the dog barking sound works well for me. I haven't seen him back since.

      By the way, do you use YOLO for the video object detection? Do you mind to share a little more information on how to do that?

      Thanks-

      Last edited by BeyondKoi; 10-25-2020 at 09:09 PM.

    7. #7
      SJK is offline Supporting Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Aug 2020
      Location
      Gig Harbor, WA
      Posts
      26
      Who knew there was so much high-tech fun to be had keeping koi!

    8. #8
      two_wheeled is offline Senior Member
      is TGIF
       
      Feeling:
      Meh
       
      Join Date
      Nov 2017
      Location
      Phoenix, Arizona
      Posts
      367
      This is a very interesting idea. How do you determine if the object detected is a heron? Would a human or a dog or a raven be identified differently?
      I had a good chuckle at the idea of a drone flying around the pond when a heron was detected. Reminded me of this scene. https://www.youtube.com/watch?v=heMboVN12r0
      -Steve in Phx.
      Novice Extraordinaire

    9. #9
      icu2's Avatar
      icu2 is online now Administrator ~ WWKC Treasurer
      is playing on Koiphen
       
      Feeling:
      Friendly
       
      Join Date
      Feb 2007
      Location
      Poulsbo, WA
      Posts
      31,758
      Quote Originally Posted by matiasklein View Post
      Thanks! Agree with you on more aggressive countermeasures. I suspect bird will get used to sound and eventually ignore it. I'll probably end up flying a small drone with blinking lights around the pond whenever the heron is detected, but for now I'm starting with less complex countermeasures like sound. Would welcome thoughts on what folks think would work best.
      This would really be amazing to see. I've flown a lot of flights via drone that are controlled via software but never
      remote start, take off, and land, and power off.
      --Steve


      Control your own destiny or someone else will.

      ---Jack Welch

    10. #10
      spkennyva is offline Senior Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jan 2012
      Location
      Suffolk, VA
      Posts
      106
      Quote Originally Posted by two_wheeled View Post
      This is a very interesting idea. How do you determine if the object detected is a heron? Would a human or a dog or a raven be identified differently?
      It is explained in post #5 with: "I'm using a custom neural network that was specifically trained on a heron dataset". Which means (I'm giving my own interpretation) he used a set of heron images to train a neural net which is run on a server where the raw video is streamed from the remote camera. I suspect that he acquired a large dataset of heron images online to improve the accuracy of the system, so that humans and other off-target critters would not trigger the system. I attempted this too, but not nearly with the same results as shown here - this system is VERY nice. There is a large dataset of images out there, including herons, so getting the raw data is not the issue. However, if you're outside the world of AI and computer programming (tensorflow, python, docker, etc), then to reproduce this would take a fair amount of effort. Someone fully immersed in these technologies can probably do this with relative ease.

    11. #11
      spkennyva is offline Senior Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jan 2012
      Location
      Suffolk, VA
      Posts
      106
      Quote Originally Posted by matiasklein View Post
      Would welcome thoughts on what folks think would work best.
      I've found that the "scarecrow" sprinkler system is pretty robust. You could easily control a relay that opens a water valve connected to a sprinkler. Anything that makes noise and hits them (like water) should really work.

    12. #12
      BeyondKoi's Avatar
      BeyondKoi is offline Supporting Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      May 2010
      Location
      Fairfax, VA
      Posts
      304
      Quote Originally Posted by spkennyva View Post
      I've found that the "scarecrow" sprinkler system is pretty robust. You could easily control a relay that opens a water valve connected to a sprinkler. Anything that makes noise and hits them (like water) should really work.
      My heron was laughing at the "scarecrow" sprinkler He walked right passed the sprinkler while it was still spraying! The dog barking sounds triggered at the right time was the affective way to scare him.

    13. #13
      matiasklein is offline Junior Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jun 2017
      Location
      Philadelphia, PA
      Posts
      8
      Quote Originally Posted by spkennyva View Post
      It is explained in post #5 with: "I'm using a custom neural network that was specifically trained on a heron dataset". Which means (I'm giving my own interpretation) he used a set of heron images to train a neural net which is run on a server where the raw video is streamed from the remote camera. I suspect that he acquired a large dataset of heron images online to improve the accuracy of the system, so that humans and other off-target critters would not trigger the system. I attempted this too, but not nearly with the same results as shown here - this system is VERY nice. There is a large dataset of images out there, including herons, so getting the raw data is not the issue. However, if you're outside the world of AI and computer programming (tensorflow, python, docker, etc), then to reproduce this would take a fair amount of effort. Someone fully immersed in these technologies can probably do this with relative ease.
      You got it. I have a proprietary dataset of heron images that I have amassed and I executed a transfer learning processes using Yolo v3. Here is an article that explains how to do the object detection piece.

      It is definitely not trivial to execute this as an end-to-end solution, as building the detection neural net is just one piece of the automation necessary. Fortunately, my company builds an AI server for large companies, so I also have all the pieces necessary to run this as a production system at my home.

      I'm exploring whether this can be integrated into a single smart camera, so it could be sold to other pond owners that want a simple, all-in-one device to effectively deter these pests. I think it's probably a very niche market, but I have this problem, so I assume others do too.

      I have read online that heron decoys, sprinklers, crocodile heads, spinners, etc. all seem to only be partially effective. However, I believe 24/7 monitoring with active countermeasures will be 100% effective.

      For fun, here is another video of the detector in action yesterday. Amazing how camouflaged this bird is with my waterfall.


    14. #14
      spkennyva is offline Senior Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jan 2012
      Location
      Suffolk, VA
      Posts
      106
      Outstanding! The ability of your net to discriminate the heron against the tree is awesome. Really nice job!!

    15. #15
      matiasklein is offline Junior Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jun 2017
      Location
      Philadelphia, PA
      Posts
      8
      Thanks. That is a funny clip of the heat seeking bullet!

      To answer your question two_wheeled, the neural network is specifically trained to identify herons in video. It is not looking for motion, and it will not falsely identify a person, dog, or other animal as a heron. It is possible for it to false alarm, but I don't have enough experience running it yet to tell you how often that happens.

    16. #16
      KoiRun's Avatar
      KoiRun is online now Senior Member
      This user has no status.
       
      Feeling:
      Happy
       
      Join Date
      Nov 2015
      Location
      Mississauga, Ontario
      Posts
      2,125
      Quote Originally Posted by matiasklein View Post
      ...less complex countermeasures... Would welcome thoughts on what folks think would work best.
      Ha ha, I myself prefer proven, low-tech, reliable methods:

      https://youtu.be/ahiQ-yZi-pk

      Last edited by KoiRun; 4 Weeks Ago at 05:21 PM.
      Time you enjoy wasting is not wasted time. ---- Marthe Troly-Curtin

      KoiRun on YouTube, latest video:
      https://www.youtube.com/watch?v=PmiHmpbWEOY&t=170s


    17. #17
      Fishmover is online now Supporting Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jul 2020
      Location
      indiana
      Posts
      354
      Quote Originally Posted by KoiRun View Post
      Ha ha, I myself prefer proven, low-tech, reliable methods:

      https://youtu.be/ahiQ-yZi-pk

      Anyone know where I can get a good price on a lion. Preferably house broken
      Last edited by Fishmover; 4 Weeks Ago at 07:15 PM.


      Been ponding for a while, with lots of successes, and lots of failures.

    18. #18
      spkennyva is offline Senior Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jan 2012
      Location
      Suffolk, VA
      Posts
      106
      Quote Originally Posted by BeyondKoi View Post
      My heron was laughing at the "scarecrow" sprinkler He walked right passed the sprinkler while it was still spraying! The dog barking sounds triggered at the right time was the affective way to scare him.
      Wow, I've been using the scarecrow for over 6 years and find it highly effective.

    19. #19
      KoiFan84 is offline Senior Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jul 2019
      Location
      USA
      Posts
      183
      Quote Originally Posted by matiasklein View Post
      A great blue heron has been eating my favorite koi. Even my ever vigilant watchdog can't consistently detect this bird since it a silent, stealth killer. The heron has also killed koi through my pond net.

      I had enough, so I created an artificial intelligence program to detect and deploy countermeasures so I can finally remove my pond net and relax.

      Check out this video of a great blue heron being detected in the middle of the night when I was sleeping. Let me know what you think!

      What countermeasures?

    20. #20
      matiasklein is offline Junior Member
      This user has no status.
       
      Feeling:
      ----
       
      Join Date
      Jun 2017
      Location
      Philadelphia, PA
      Posts
      8
      Quote Originally Posted by KoiFan84 View Post
      What countermeasures?
      Sound of dog barking and whiffle balls shooting. Check it out, it works perfect!


    Page 1 of 2 12 LastLast

    Tags for this Thread

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •