Q & A with Morgan Carey (Real Estate Webmasters) about Indexable IDX
February 23, 2010
By: Tony Kawaguchi
Author's Website: http://www.alohatony.com/
Today’s feature: An interview with Real Estate Webmasters CEO Morgan Carey about the SEO benefits of indexable IDX. He reveals hard data on the resulting increased traffic, and the ability of individual agents to compete in the elusive quest for search results.
There’s been a lot of chatter about some of the newer listing scrapers and IDX wordpress plugins and what they can do, and what the value is or isn’t of having a spiderable IDX products.
I’m interested in this topic because I got a spiderable IDX in May of 2009 from Real Estate Webmasters. So far the numbers look good, but I want to know what the potential is, and whether people’s concerns about the topic have any validity. So, for my own research (and yours), I’ve decided to do a case study on my blog that looks at the effects of having a spiderable IDX on my site, and on sites that have higher authority than mine. I’d like to see where my traffic will be in a year or two!
Also, after reading all the posts at Active Rain and Geek Estate Blog, I decided to speak with Morgan Carey of Real Estate Webmasters to see if he could answer some of the most common question/concerns regarding spiderable IDX.
Question: If I have an indexable IDX, will I get a duplicate content penalty if others also have an indexable IDX?
Answer: No, that’s a myth. From Google itself: “Let’s put this to bed once and for all, folks: There’s no such thing as a ‘duplicate content penalty.’ At least, not in the way most people mean when they say that. Duplicate content on a site is not grounds for action on that site unless it appears that the intent of the duplicate content is to be deceptive and manipulate search engine results.” Furthermore, although it is true that everyone is starting with the same “data set” – not everyone will implement it as effectively or in the same manner. Diversification and creativity makes having this kind of data hugely valuable if leveraged correctly.
Question: Will a spiderable IDX help my site compete with the big guns (Redfin, Trulia, Zillow etc.)?
Answer: Yes of course. It’s weird – folks seem to be thinking now that the “cat is out of the bag” about spiderable IDX (we have been doing it for over 6 years btw, it’s hardly new) that there is “no point” in having spiderable IDX because companies like Trulia or Zillow also have spiderable listings (many powered directly by MLS feeds),so don’t even try to compete. The fact of the matter is – there is a very small amount of these super powerful aggregators out there, and there are 10 spots on the first page of a search engine. Even if you couldn’t outdo these sites, why not be in the fray? #4 for any keyword on the first page is really not a bad place to be. That being said – although these sites from a “domain” perspective are very powerful – they must distribute this power to their most popular / desirable targets and cannot possibly dominate or even focus on all listings in all areas. Many times just by having a spiderable IDX and a decent site, our customers easily outrank the big guns on address and MLS # related searches. One thing is for sure – without spiderable IDX – you aren’t even on the field, let alone in the game.
Question: I have a high authority site, great rankings – if I add spiderable IDX how will this change my stats?
Answer: Well, check out Jay Thompson’s case study, or take Gary Ashton‘s site. Gary got a spiderable IDX at the beginning of January, this year. You’ll notice the massive jump in the number of pageviews on his site:
- Gary Ashton’s Page Views
His traffic’s also increased over the previous month by about 45%, while his bounce rate has dropped by 36%. Users are spending about 2 1/2 times as long on the site, and they’re exploring an average of 8 pages on the site, whereas before when he had an iframed solution, people were only visiting about 2 pages on the site.
Question: I have spiderable IDX and I only have a few hundred pages indexed – why not all of them?
Answer: Chances are you do not have the authority (domain) required to keep that many pages indexed. A solution is to improve your overall pagerank, and also work on your internal linking and architecture as much as possible to leverage the pagerank you do have. Still not the end of the world though – that’s probably 350 more pages driving traffic to your site than you would have had without it. Chin up – keep building links and working on your site structure, and your pagecount will reward you with a ton of long tail traffic over time.
Question: There are many websites (with as much or more pagerank as me) in my area that have spiderable IDX. How can I compete?
Answer: Again, build your authority / pagerank is step one – whoever has the most juice (domain) has the best chance. However, there is something to be said for differentiation & augmentation. If you can come up with creative ways to rewrite your information using algorithms (titles, meta data etc.) so as to provide a “different look” (layout) and additional information on the page, then your chances are far better to generate additional long tail traffic than those simply using “out of the box” spiderable IDX. Also try to find unique ways of getting your users involved – does your MLS allow after-listing comments? Why not have users contribute comments about their favorite listings, thereby adding valuable visitor-generated, “unique” content to the page?
Here’s a question brought up by Jeff Corbett: Does the SEO value evaporate, since everyone will effectively have the same content?
Answer: I would say that everyone needs to make sure that outside their IDX data, their site has unique, quality content, period. This will help differentiate your site from the rest of the pack. As for the listings themselves, as I mentioned above, you can modify your titles and meta data to make your listings different from everyone else’s. Focus on the basics of good SEO, and the listings will only help your site in the rankings–as well as improve user experience. BTW – unless you are in a market like Austin, TX where everyone seems to be on the ball (example http://www.jimolenbush.com), then chances are you only have to compete with 1, maybe 2 sites – again, although it’s not exactly a secret, it’s also not like anyone is taking advantage. How long exactly do you want to wait before everyone but you does have spiderable IDX and there is actually some competition? Life rewards originators.
Question: Is there anything people should be aware of before they decide to get a spiderable IDX?
Answer: Realtors should stay away from IDX vendors with RSS feeds – these people are violating the NAR’s guidelines with respect to their obligation as vendors to protect IDX data from scraping or misappropriation. If the MLS board was informed (and educated as to why) RSS should not be allowed, chances are they would pull the feed. Furthermore, any solutions (such as the DS IDX wordpress plugin Jay Thompson references in his post) – should be avoided as well. My feeling is that because these plugins put complete control into the hands of untrained webmasters with no regard for regulations or compliance, the various boards will very soon pull the feeds (and perhaps the vendor agreements) of those not adhering to the quality standards expected of IDX vendors. The boards already have enough problems with members reporting members on items that are not even real issues. Imagine what will happen to a board’s compliance staff once these IDX’s (now in the control of untrained webmasters / Realtors) start gaining popularity. The MLS board is going to have so many legitimate complaints, they will virtually become buried. They obviously don’t want that, which is why they have “approved” vendors and a compliance process in the first place. Any spiderable IDX company however that does have a compliance department, and is responsible and “accountable” with regards to the MLS boards’ terms of use should be fine.
- Stumble it!
- Categories: IDX, Search, Wordpress
Enjoy this post? Subscribe to the Geek Estate Blog feed or get updates via e-mail
Comments
View Comments
Morgan Carey on February 23, 2010 12:24 pm
Thanks for posting Tony – I wanted to take a moment to invite others that may have questions regarding spiderable IDX to chime in here – clearly a topic folks are interested in, but one that with all the technical complexities can be quite overwhelming for your day to day (non programmer) types.
Erika Eaton on February 23, 2010 1:56 pm
I have the DS IDX WordPress plugin you mentioned in your last response, and am now somewhat concerned. Can you please explain why it's an issue.
Todd Carpenter on February 23, 2010 2:35 pm
Morgan, I think Diverse Solutions has been providing an RSS feeds for at least a year. Can you identify any situations where those feeds have lead to actual complaints?
tony on February 23, 2010 2:43 pm
One very likely issue that will come up when you put an RSS feed out for IDX – anyone can do anything, literally anything with that data. Scrapers can reuse it, publish it, copy it, etc. By definition, it makes the IDX data completely open to everyone on earth to manipulate.
Justin LaJoie on February 23, 2010 2:49 pm
Great idea for a post it’s been quite the topic lately and I agree with most of what Morgan says here. His answers are very similar to what I would have said. The obvious problem I have is with that last answer. If find it humorous when a competitor tries to bash us by giving false information as if they have some insight into our company. All it does is make them look like a fool when the truth comes out.
Let me start off by saying Diverse Solutions takes compliance very seriously. We have been working directly with the MLSs across the nation in regards to compliance on all of our products since day one. To say that we do not maintain compliance in our products and that various boards will pull our feeds is a complete fallacy and just a cheap way for you to try and discredit us. It may work on the phone for you when clients only get to hear one side of the story but it won’t work here.
RSS feeds were introduced for feed readers for use by consumers as an alternative to receiving new property updates view email. There is a limited amount of information on each property due to compliance and consumer strategy. There are also only a set number of results that we give at any one time. Both of these are set conservatively by us but changed on a case by case basis per the MLSs request. In fact there are even a couple MLSs that didn’t like the RSS feeds and in turn we do not offer them in those areas. However, recently we have seen in some cases were agents have gotten creative and used them to put updated listings into blog posts. We do not advocate this nor do we promote this. I have also seen an agent set the same type of thing up but instead of using an RSS feed they used email updates. Does that mean we should disallow email updates? No. At a certain point the agent does have to be responsible and be accountable to the MLS.
As far as dsIDXpress is concerned you couldn’t be farther from the truth. Compliance is the #1 rule in IDX. The way we designed our IDX wordpress plugin is so that the agent never is in control of the data. The data comes from our server already compliant in the appropriate format with all the disclaimers needed. That way they can put it anywhere they want and not have any worries. We have worked hand in hand with many MLSs that have more complex rules so that dsIDXpress is complaint and we have their blessing.
Note: If any Diverse Solutions customer has any questions about compliance give us a call.
mlbroadcast on February 23, 2010 3:11 pm
Tony. The ability to scrape MLS data probably exists in a number of ways. I am also interested in hearing an answer to Todd's question. Have there been instances of complaints regarding the use of RSS feeds in IDX?
Tony on February 23, 2010 3:11 pm
Justin
I understand your point and that you try hard to stay in compliance. While i think there are still issues there, i”m far more concerned about the next level of people who can take your RSS feed – and by definition – do whatever they want with it. You have no control over what people do with your feeds. I'm not talking realtors, i'm talking anybody on earth who knows how to manipulate a feed.
There is no possible way for you to control that. Am I missing something?
Tony on February 23, 2010 3:29 pm
Yes i understand the ability for scraping exists – but with an RSS feed, it's no longer scraping is it? It's just taking the feed and doing whatever you want, which is not scraping. see where this is going? you can't even know who is doing it, and where they publish it.
jameswheelock on February 23, 2010 3:30 pm
I have posted here before and not sure as to why I am being censored.
drewmeyers on February 23, 2010 3:36 pm
James-
Sorry, your comment got stuck in the disqus SPAM queue for some reason — but I just approved it. I assure you, you are not being censored by any human behind the scenes on Geek Estate
Morgan Carey on February 23, 2010 3:40 pm
That's a great point Tony – you are completely correct, Diverse has absolutely no control over what folks who have access to the RSS feeds they provide (which is everyone including scrapers and data miners) – so although Justin claims (and I believe him) that Diverse tries to maintain compliance in their search solution itself, they are not maintaining compiance in their RSS feeds, nor are they able to control or enforce compliance in anyone using their open RSS feeds. It is the RESPONSIBILITY OF THE VENDOR to protect the data – and putting it out there in an open RSS feed to be scraped is not responsible, nor compliant. You already admitted you are aware of your own customers scraping the data for their blog posts and stated that you did not approve of it. So you know about it, you know it's a problem, why then do you not take measures to stop it?
Here is a quote from NAR btw Cliff Niersbach of NAR (who is quoting IDX policy)
“Participants (and vendors) must protect IDX information from misappropriation by employing reasonable efforts to monitor and prevent 'scraping' or other unauthorized accessing, reproduction, or other use of the MLS database.”
Diverse Solution violates this by publishing in an easily findable format RSS feeds, knowing full well that “their customers” and others can easily scrape and repurpose this data. Justin your defense about RSS readers being introduced for readers (and that being their original intent of publishing RSS feeds) is laughable – you know damn well what RSS feeds are used for these days, and it ain't joe public getting property updates, it's webmasters and scrapers using RSS to publish content to their sites in any (non compliant) way they see fit.
drewmeyers on February 23, 2010 3:42 pm
Tony-
RSS feeds are governed by terms of use. Sure, abuse is possible w/ a RSS feed just like it's possible to scrape a site. But just because someone has an RSS feed does not mean anyone else can do anything (legally) they want with it.
mlbroadcast on February 23, 2010 3:50 pm
I hope this doesn't evolve into a competition squabble, however, the point/counterpoint of the issue is very important to real estate professionals using IDX technology solutions. If in doubt, check with your MLS provider as to compliance on the part of your IDX solution provider.
Tony on February 23, 2010 3:53 pm
James – can you specify a little about the long term costs of the DS product? I am not familiar, but since you brought it up…
drewmeyers on February 23, 2010 3:54 pm
Morgan-
While I have a huge amount of respect for the work you've done with IDX, it should be stated that Geek Estate is not the place for any sort of “competitor bashing” – particularly based in incorrect information. Your product should speak for itself without having to go to these levels.
I'm disappointed to see you employ these tactics to grow your business.
justinlajoie on February 23, 2010 4:02 pm
@ Tony – We limit the RSS feeds to the first 25-50 results and rate limit the total number sent throughout the day if they have multiple RSS feeds as each feed is unique. It would be impossible to index all of an MLS this way. As it is a novice programmer can scrape an HTML site like yours just as easy as an RSS feed. The rule of thumb is. If you can see it then someone can scrape it. It’s just how long is it going to take and how hard is it going to be to scrape.
Tony on February 23, 2010 4:02 pm
Drew, you're right, this is not the place to bash competitors, but I don't think that was what Morgan was trying to do in his interview.
I don't know his motivation, but this interview post idea was for me, not for him. It was all about the tech side, which is exactly what readers here want to know about.
Anyway, sorry for causing some offense. Though, I have to admit, it sort of shows how the discussion is all the more important.
Maybe I should interview Justin about the RSS feed as a follow up post? Or maybe he would prefer someone else do it now
Morgan Carey on February 23, 2010 4:13 pm
Actual complaints? No – I am not a member of any MLS board nor am I staff at DS so I would not be privy to such complaints – that being said, that fact does not disqualify me from understanding IDX rules / regs and pointing out that they are in fact a violation even if noone has had their hand slapped yet (again, I am not saying they have or haven't because I don't know) – Justin from DS has already stated below that several boards (likely the more web savvy ones) disallowed RSS feeds altogether – wonder why they would do that?
Morgan Carey on February 23, 2010 4:16 pm
Drew – “Legally” has nothing to do with scapers – by their very nature they are willing to commit illegal acts with copy written materials. A vendors obligation is to protect the data within “reasonable measures” from scrapers. Obviously noone is concerned here with folks that follow the rules – if every did that, there would be no issue and MLS boards wouldn't need vendors, they would just make their data feeds public along with a TOS stating who could use them and under what conditions.
Drew on February 23, 2010 4:20 pm
Justin, I am curious about DS compliance. SoCal MLS informed me that they require that the listing agent DRE license number be listed on the detail pages along with broker/agent info. They made me do it. If true, I notice that you are not in compliance. Perhaps an oversite, maybe I am misinformed or since you are a preferred vendor it is overlooked? I have always wondered about that as other vendors do show the DRE number.
Morgan Carey on February 23, 2010 4:22 pm
Competitor bashing is not my intent at all Drew, I already addressed that above (and if you read the entire interview there was only one mention of any competitor, I meant to use it as an example because it was a “current” item and one that had been referenced as an example post from Jay above) – perhaps I could have just said “beware of vendors that put the control of IDX data in the hands of untrained / non accountable programmers” – and left the mention of DS out of it entirely. Again, this was meant to be a discussion regarding the merits of spiderable IDX, and seemed like a good place to have it (lots of people have LOTS of questions about it) – I would just as soon see less DS talk and more discussion relating to the non vendor items which to be fair, cover 6 of the 7 points.
jameswheelock on February 23, 2010 4:24 pm
Tony,
I don't want to get into a discussion about definitions, but using data from another site without permission by RSS feed is no less scraping than writing code to pull it directly from source code.
I think what you are getting at is that RSS feeds make it much easier to scrap than normal and that is true. However, the RSS feeds being available is a huge benefit to consumers. I have many people comment on how nice it is that my IDX has that feature.
Now as to it being more difficult to track a scraper using the RSS feed over writing code to pull it from source that is just not true. In fact I would say that there are ways to make it easier to tract a scraper using a RSS feed.
Morgan Carey on February 23, 2010 4:27 pm
Incidentally Justin, I just posted a request for sample data and installed your plugin – as I said, I made my statement based on what I was told users could do with your data, but I do owe it to you to verify it first hand. Should you decide to approve my sample data request, I will investigate and if I owe you an apology (or have made any false statements) – I am certainly happy to do so.
drewmeyers on February 23, 2010 4:29 pm
I'm glad that wasn't your intent, but I think it should have been worded differently to make that more clear. My concern was with this statement since it said specifically avoid working with DS:
“Furthermore, any solutions (such as the DS IDX wordpress plugin Jay Thompson references in his post) – should be avoided as well. My feeling is that because these plugins put complete control into the hands of untrained webmasters with no regard for regulations or compliance, the various boards will very soon pull the feeds (and perhaps the vendor agreements) of those not adhering to the quality standards expected of IDX vendors.”
jameswheelock on February 23, 2010 4:30 pm
Upfront costs are hugely less with Diverse Solutions, but monthly cost is about three time higher. Therefore over the years it can get more expensive to use DS than REW assuming no modifications.
Although the few ways DS will allow modifications they charge must less to do them especially if they have been done before.
I like I said before I have mixed feelings about the two products because REW is missing some great features that DS has and they would likely be very expensive additions.
Overall I think REW and DS are the top two providers in the entire market place.
jameswheelock on February 23, 2010 4:30 pm
Good to know and thank you.
jameswheelock on February 23, 2010 4:34 pm
@Justin and Tony – I would be more than happy to interview Justin or Robert about the Diverse Solutions product.
Justin LaJoie on February 23, 2010 4:44 pm
We include the DRE# on our dsIDXpress and our dsSearchAgent IDX products for SocalMLS. There are a handle full of people that are using an old product that does not have it and for technical reason can not. But the MLS is aware and is handling this as the agent/broker should be including the DRE on their website if they want to continue using it.
Justin LaJoie on February 23, 2010 4:46 pm
Go for it. It's free to try.
jameswheelock on February 23, 2010 4:47 pm
Compliance questions would be better directed at Robert Larsen. I think that Mr. Larsen should be invited to the conversation to discuss Diverse Solutions compliance issues.
Morgan Carey on February 23, 2010 4:55 pm
Thanks, I have installed and activated it – I'll post my findings later tonight (Drew Myers would you like me to post it somewhere else so as to not clog a productive discussion with a vendor specific item? – let me know)
Morgan Carey on February 23, 2010 4:58 pm
Fair enough (I honestly hadn't given it a second though) if you would not mind (with my permission, and I think Tony's as well) perhaps remove the (such as the DS IDX wordpress plugin Jay Thompson references in his post) -
I still stand by my statement however, that if a vendor (who it is doesn't matter) simply hands data over to the webmaster (who is not obligated by compliance or at the very least, not educated in the matter) should be avoided.
I would actually think that Justin would agree with that statement – no?
drewmeyers on February 23, 2010 4:59 pm
Morgan-
I appreciate you digging into this to verify the facts. You're welcome to post your findings here.
drewmeyers on February 23, 2010 5:08 pm
I just put a strikethrough that sentence, but left it in there since much of the comment thread is talking about that sentence. Regardless, I appreciate your willingness to address this. Everyone makes mistakes, even me
Morgan Carey on February 23, 2010 5:14 pm
Thanks Drew – I appreciate it.
tony on February 23, 2010 5:38 pm
a very important question for me is – why did some MLS boards decide not to approve the RSS feed? Can you address this Justin?
Morgan Carey on February 23, 2010 5:41 pm
Incidentally, I would like to offer a defense for the statement that everyone seemed to get so mad at for me making (without verification at least) – I downloaded the plugin and was easily able to make non compliant modifications to the output of the DS product (here is a sample screenshot http://www.realestatewebmasters.com/blogs/uploa... note I added my phone number, my logo super imposed over the phone and another non compliant item)
So to the user who said:
“you are unable to alter any of the IDX pages it displays. Further even in the cases were you have some control over what gets displayed it still comes with all disclaimers and you cannot leave off the credit to the brokers.”
You were completely incorrect – they are easily edited and a user can do whatever they want with those areas including removing any required disclaimers or changing any data you want. Am I still the bad guy here?
Morgan Carey on February 23, 2010 5:43 pm
That URL doesn't work but somehow it became an attachment, click the attached file (not the URL) if you are trying to see the sample.
mlbroadcast on February 23, 2010 3:44 pm
I can verify that as well, we had the same issue with Justin's comment. We may need to look at our comment system, Drew is more familiar with it than I am, so I'll work with him to improve it.
randy on February 23, 2010 6:04 pm
@morgan I dont know about the rest of you, but the way you're going about this, with a competitor, seems a bit shady.
randy on February 23, 2010 6:18 pm
In addition, doesn't it seem like diverse would just shut off your site if you did something like that. why would they allow you do continue to do anything that violates their agreement with the MLS. your argument seems pretty petty and thin in my own opinion. as a framed IDX customer I could do any number of things to my website that angers the MLS via my vendor. I think it's silly to go around pointing out ways you can piss off an MLS like this. I'm not even sure why we're all arguing about it. I'll say this, and I'm done:
This seems like a huge non issue, the only one complaining is morgan. Justin said they got MLSs to sign off, enough said.
Jay Thompson on February 23, 2010 6:27 pm
“- they are easily edited and a user can do whatever they want with those areas”
Easily edited by someone like yourself with crazy coding skills, or easily edited by the average real estate agent?
I consider myself above average with regard to web skills and have no idea how to edit the DS plugin pages.
Morgan Carey on February 23, 2010 6:56 pm
Randy – I agree this should be a non issue, that last comment in an otherwise positive article was blown WAY out of proportion – I didn't agree to do this based on constantly talking about DS – you think I want to give them this much attention?
The fact of the matter is, I got jumped on from a bunch of folks because of the fact that I included DS in that last line (something I truly regret doing) – but now it's almost like I am pot committed in terms of defending myself – both Justin and James basically said I was full of it and claimed that what I had posted above was false (they basically called me a liar) – now again, although I completely agree that I probably could have left out the DS reference – the fact of the matter is, my statement was completely correct, and now I feel compelled to defend myself – know what I mean?
Perhaps I should just cool off for a bit because I really don't want to go down this path – if anyone has any SEO related questions with respect to spiderable IDX (which is why I came here) I am happy to answer them, this DS BS I am done with.
morgan carey on February 23, 2010 7:01 pm
Jay you just wrote:
“I can not change, alter, or edit any of the compliance text generated by the DS plugin,”
And you are wrong – I just posted a screen shot of my DS account and anything and everything on the DS page is editable, you just have to go through the files with a simple prag match as opposed to via the plugin – just because you haven't looked for it, or don't realize how easy it is to do, doesn't change the fact that they are completely editable.
In this solution (the DS Worpress solution) the only control DS has is whether or not to give you the data feed, after that, everything is up to whoever owns the site – that is the part I have a problem with – DS is no longer an IDX vendor, they are simply a reseller of the feed. I feel this violations their status as a vendor, others don't agree – fair enough.
Anyways – lets all just go about our merry lives and assume noone will use this application based on their moral integrity, and start talking about spiderable IDX shall we?
Jeff manson on February 23, 2010 7:04 pm
We have been indexing pages for years and it definitely will increase your long tail. Still think your better off on getting your communities to rank if you do not have a lot of Authority to pass around. More users are searching for the communities than single addresses.
Good post until Morgan took a shot at DS
Morgan Carey on February 23, 2010 7:04 pm
Fair enough Jay
– but seriously, how many agents have the skills you have? (Especially if they are messing with wordpress) most will have to hire a programmer – and believe me when I tell you, for “any” programmer – editing these files is not hard at all.
Morgan Carey on February 23, 2010 7:07 pm
Believe me Randy, if I wanted to intentionally target DS, I would not do it on a site that I do not have editorial control over – this whole thing kinda sort just “happened” (again, for like the millionth time, I wish I had just left it generic instead of mentioning DS because this whole post has been unfortunately derailed)
Jay Thompson on February 23, 2010 7:12 pm
For what it's worth, I posted that comment before you posted you could easily change the DS page. This comment was caught in spam.
I have no clue what a “simple prag match” is. My idea of coding is copy/pasting public domain code I find and struggling to get it to work on my sites. I'm not a coder by any stretch of the imagination. What is easy for you, isn't easy for many people. so **I** can't change a DS plugin page. You can, but I can't.
Keeping in mind I have basically zero programming skills, couldn't someone also take a REW page and change it to be non-compliant?
I just copied the source code of one of my REW pages and changed the ARMLS logo to a REW logo. Granted, this page has no styling applied to it, but it DOES render the page non-compliant.
http://www.phoenixrealestateguy.com/images/repl...
Morgan Carey on February 23, 2010 7:13 pm
Lol fair enough – and thanks for trying to bring it back Jeff. It's definitely true that you don't want to “dilute the juice” with to many spiderable IDX pages if you are like a PR 2 or something (Real PR, not the Toolbar stuff)
It's a gradual thing, let a few communities out at a time (perhaps just residential and perhaps just between some desired price ranges) until you are able to grow your authority to the level it needs to be to support a larger page volume.
Of course, targeting communities requires far more umph than targeting addresses so it's kind of a double edged sword – day one you can rank for an addy in most markets, where it can be a couple of months of auth building before you can start showing for communities.
Jay Thompson on February 23, 2010 7:15 pm
Oh, and I agree, let's get back to talking spiderable IDX…
Morgan Carey on February 23, 2010 7:30 pm
You did that manually did you not? That's not something you did with the REW IDX, that's something “you did” – two completely different things (heck you could have just done that in photoshop) – but the point needs to be made that you weren't able to compromise the REW IDX nor are you using it in any way to generate that file – I can take a screenshot of Google or post their source code somewhere else too – but that doesn't mean I have special access to leverage their technologies in an unauthorized manner.
Jay Thompson on February 23, 2010 7:33 pm
My photoshop skills are on the same level as my coding skills…
Yes, that screen shot was done in a very manual way. Literally copied source code and changed the logo image. I told you I was a mad coder!
Justin LaJoie on February 23, 2010 9:06 pm
And this is the last I will say on this topic
@Morgan – I know that at this point your are just trying to save face and I understand that. We all wish we hadn't said something at one time or another but you have got to work on your PR skills. While I appreciate you checking out our product and giving your unbiased feedback it doesn't really mater to much to us. Having a competitor hack our wordpress IDX to put a logo and some additional words, take screenshots then tell everyone that it was so easy doesn't really hold that much weight. I can guarantee that if you give us 15 minutes with your idx website that we can either change the CSS and/or inject some HTML somewhere to make it uncompliant too. The fact of the matter is that the consumer just doesn't do that. If we find anyone that does we deal them on a case by case basis. Including shutting off their access. Why are we getting into progamming debate here when very few would understand us and it just doesn't mean anything. It would be just two vendors pounding on their chest like a bunch of idiots. Lets save that for that bar at Inman SF this summer. I'll buy the first drink. As far as our compliance is concerned, let us worry about that we've been doing just fine.
Morgan Carey on February 23, 2010 10:06 pm
Believe me Justin, I wish that someone had come along, read this thing and went – wow spiderable IDX is powerful! Then went and found either of us, instead of someone jumping on the DS reference (I still think it was blown WAY out of proportion) – and work on my PR skills – are you freakin kidding me, I've been wishing a good PR person would fall in my lap for years – I like building and SEO'n stuff – not trying to work my way in (or out) of arguments online – obviously I have a knack for working my way in
Does ANYONE want to talk about the pros / cons or SEO advantages of spiderable IDX – or should we change the title of this post to “some guy named posted an article and some other guy called out some other guy's product”
Kind of like a train wreck though – (not just because it seems to be hard to survive) but also very hard not to watch heh – ah the internet.
Morgan Carey on February 23, 2010 10:08 pm
Maybe we can work out a deal – I'll code your stuff, and you blog for me – I SUCK at it!
Realtors on February 23, 2010 10:17 pm
Thanks for the info about the IDX. Just encountered a few problems and I think it'll work out this time.
NancyD on February 23, 2010 11:54 pm
Wow, this is quite some conversation! The interesting thing to me is that Diverse Solutions and REW are the two IDX solutions (and website providers) that one of my clients has boiled down her decision to. We like aspects of both companies. It has been a very difficult decision to make.
What I don't want either of you gentlemen to forget is that customer service and ease of use are of utmost importance to your clients. Again, you each have a fabulous product… with strengths and weaknesses that “the other guy” makes up for. What one of you lacks is good customer service. That may make all the difference in our decision as to who to go with.
Just my 2 cents worth… for what it's worth.
tony on February 24, 2010 10:12 am
Nancy – one thing i like about REW is that it's big enough to have a lot of people to help me at anytime. everytime i call, email, or post in the forum, i get a pretty quick response.
on the other hand, it's a big enough company that sometime i feel like a little tiny fish in a pond full of bigger fish…
but i still vote for REW
Top 10 real estate posts of the day for 2/24/2010 : Tempe real estate and free home search on February 24, 2010 8:48 am
[...] Q & A with Morgan Carey (Real Estate Webmasters) about Indexable IDX – Indexable IDX results, do they hurt you? Do they help you? If you are not having some sort of [...]
Tony on February 24, 2010 10:49 am
Justin, can you answer my question from earlier – why did some MLS boards choose not to participate in RSS feeds?
rongoodman on February 24, 2010 1:15 pm
All generally true. A spiderable IDX is a clear advantage.
RE: “… and they’re exploring an average of 8 pages on the site, whereas before when he had an iframed solution, people were only visiting about 2 pages on the site.”
This statement is not really accurate or relevant. With an IFramed IDX, Google Analytics will only count the main page containing the IFrame as a single access, and will not count how the IFramed IDX content is viewed within that page. This is primarily because the IFramed IDX urls do not contain the site-specific Analytics code to have those pages counted. A user can actually be looking at dozens or hundreds of pages within the IFramed page per visit, but Analytics will never see or count that usage. Therefore, using this statistic for a comparison between spiderable and non-spiderable IDX site is misleading and not appropriate.
I agree strongly that compliance issues can arise from allowing individual site owners total control over the IDX content and layout, as some open source IDX solutions do. However, if the vendors really want to protect themselves and their clients, there are ways to control and manage that. One would be to provide a “locked down” basic IDX compliance framework, residing on the vendor's servers and not accessible in any way to the site owner, that enforces vendor and webmaster compliance with MLS rules, while still allowing the webmaster some flexibility in managing the content and layout details within that framework.
Our Denver MLS is generally very diligent about compliance enforcement, but some agent webmasters, and even a couple of those with an otherwise compliant REW IDX, still manage to break the IDX display rules, like failing to display the IDX logo and listing office name on every IDX listing, including summaries, as required by our rules. Morgan, I would be happy to point out which site and why it is non-compliant if you care to contact me offline.
Bob on February 24, 2010 3:32 pm
You must have thought you were on Morgan's forum
Morgan Carey on February 24, 2010 3:48 pm
Is anyone else getting a ton of late notifications about comments that were posted 20+ hours ago? Were they stuck in moderation as well?
Morgan Carey on February 24, 2010 4:13 pm
I had to unsubscribe from this thread as every time I hit send receive I get another 20 notifications (duplications) – sorry for late responses if you leave them, will check back manually from time to time.
Jeff manson on February 24, 2010 4:16 pm
Yep – must be the commenting system is screwed up or has a hicup.
jameswheelock on February 24, 2010 4:50 pm
@Drew – I think I must have had another one of my comments thrown into the spam queue as it has been some time and it has not posted. I had a question I would really like to hear peoples take on.
Chad Barczak on February 24, 2010 4:52 pm
As fun as it is to sit back and watch REW and DS go at it, I thought I would offer my two cents on the issue for what it’s worth. First of all, any IDX vendor you are going to use has an obligation to ensure compliance for its clients and the MLS members. All of the major vendors are well aware of the rules and restrictions in regards to the display of MLS listings on behalf of the members and follow those rules as interpreted. I certainly would never imply that Justin at DS is intentionally going against any rules; as it comes down simply to the “interpretation” of the rules for each MLS.
Our company (IDX, Inc.) has not previously provided RSS for anything other than the office’s own listings due to the fact that almost all of the 350+ MLS agreements we currently have in place prohibit this type of distribution method as “interpreted” by us. By definition, RSS is a “parsable” data file, so anyone who is allowing this on anything other than their own listings, is (as Morgan points out) could be seen as a violation of a specific MLS’s rules on the surface and could be subject to fines and penalties by the MLS. Here at IDX we tend to agree (in general) with this interpretation of the rules. Now, before I get accused of saying some vendors are in violation, the way in which this RSS is delivered could be locked down to mitigate the potential of that page from getting parsed, therefore defending against this argument with the MLS. At that point, the issue really becomes and enforcement issue and this (IMHO) is completely subjective based on who you are dealing with at any particular MLS. As long as measures are put in place, whether on the page, in the firewall, or limitation of the RSS delivery to prevent the scraping and redistribution of the data, that would be a defensible argument for any IDX provider.
I do think, based solely on my own personal experience, as more and more MLS’s start to focus on enforcement, they will (unfortunately) begin to explicitly prohibit RSS feeds on any listing that is not owned by that office. Now, please understand that this is my opinion based on the current trends I see, I actually would prefer to see them specifically allow this type of distribution. In fact, that is the only reason we have not previously built a more extensive RSS feed in our own general IDX search results due to the potential legal liability we see in providing this type of feature.
Great discussion by all above, I have a great amount of respect for all of the IDX vendors out there and we all have a lot to offer in many unique and creative ways, not the least of which is indexable IDX listings, which was what prompted this entire discussion in the first place!
Cheers,
Chad
drewmeyers on February 24, 2010 8:14 pm
Yea, I think Disqus was WAYYY behind in their email notifications. I got about 50 emails today from this thread. I think they are caught up now (I hope)…
drewmeyers on February 24, 2010 8:18 pm
Weird – sorry about that. I just approved it and whitelisted you as a commenter. Hopefully it doesn't happen again.
Morgan Carey on February 24, 2010 9:05 pm
Ron – compliance issues should be sent to idx at realestatewebmasters.com, we are happy to investigate them and appreciate the heads up.
Jon on February 25, 2010 8:23 am
re: Email updates — *ironically* the RSS feed for the comments works exceptionally well
#emailupdatesaredead #longliverss #hatersbesteppin
Tony on February 25, 2010 11:22 am
Chad – great response. I agree completely. I think RSS will be banned by many of the boards soon enough. NAR should weigh in. If they let it go, great. But I doubt they will.
either way, thanks for your input.
Tony on February 25, 2010 11:22 am
Chad – great response. I agree completely. I think RSS will be banned by many of the boards soon enough. NAR should weigh in. If they let it go, great. But I doubt they will.
either way, thanks for your input.
Nozmo on February 25, 2010 12:51 pm
We are in Portland, ME where our board has a tight control of data. Our MLS data is provided by Marketlinx, which offers a very so-so product. It would be great to have access to IDX feeds at the agent level, and then use a product such as that offered by RWM. Any suggestions out there as to what arguments word with a local board entrenched in old-school thinking?
billiehillier on February 25, 2010 6:11 pm
Why is this have a strike through?
Furthermore, any solutions (such as the DS IDX wordpress plugin Jay Thompson references in his post) – should be avoided as well.
drewmeyers on February 25, 2010 6:12 pm
Bill-
Please read the comment thread for the conversation that took place as
a result of that sentence.
billiehillier on February 25, 2010 6:14 pm
Thanks – I read it in the email – too long of a day for thinking for myself.
billiehillier on February 25, 2010 6:36 pm
I have several clients I have installed the new DS product on their site and within a week we are seeing an increase in traffic and leads.
One client's broker didn't want to sign the release sort of related to the issues mentioned here because of information released on a website when the owner didn't want it published. They got sued…yada yada.
This is a very interesting post to read and even though the tone got a little heated, I am glad all the information is out there now to go through.
Now to read it again tomorrow with a fresh brain so it will make more sense.
Bob Wilson on February 26, 2010 10:53 am
Tony, give me a break. You are a client of Morgan's. Its not as if you are unbiased. Why dont you ask in your interview the danger of using his IDX and then exercising your right to freedom of speech without having your site shut down like the agent in Davis Ca this week. My gut tells me that if you wanted to be unbiased and posed these questions to all IDX providers, you wouldnt for fear of retribution.
Morgan Carey on February 26, 2010 11:29 am
Bob you are completely out of line here – the customer that just had their website shut down in Davis CA, had her site shut down for hiring a hacker and stealing REW intellectual property. She (and the hacker) are currently under investigation by the RCMP (file # 10-5102 if you care to verify) who are working with the FBI to pursue criminal charges for cyber crimes.
I would thank you and anyone else who reads this to stick to the topic and not bring in red herrings that you know nothing about nor have anything to do with the topic at hand. In fact Drew, I would request that you delete that comment – it's completely out of line.
Bob Wilson on February 26, 2010 3:55 pm
I have heard this from you before. Everyone is always out of line except you and no one knows anything but you.
drewmeyers on February 26, 2010 4:24 pm
Morgan/Bob-
I'm not going to delete anything yet, but the personal nature of this conversation needs to stop right now. Anything further along those lines will be deleted.
Morgan Carey on February 26, 2010 4:33 pm
Drew, I am surprised and quite disappointed that you allowed the first one – I mean shoot, I made the mistake of mentioning a specific vendor as an example (which I later verified and provided proof of my claim) and I got absolutely chastised – but the shoe being on the other foot seems to be completely condoned here – I certainly don't deserve (nor does Tony) to have this post being taken over by known REW haters taking non topic specific pot shots at me / Tony or anyone else – it's quite disgusting actually, and I am REALLY disappointed that you allow it.
drewmeyers on February 26, 2010 4:41 pm
Not sure how it happened, but I somehow “liked” Bob's comment even though it is exactly the type of comment that should not be repeated here on Geek Estate. I'm not sure how to unlike it w/ Disqus or I would.
drewmeyers on February 26, 2010 4:46 pm
Your comment providing further information regarding the point Bob made seems to help you and make Bob look bad, so not sure what the issue is. Note that I just commented out Bob's 2nd comment.
Morgan Carey on February 26, 2010 5:53 pm
That's the thing Drew, by allowing comments like that, it basically forces the target to defend themselves (which they should not have to do) – hence my comment about having the first one approved in the first place. Once again (and for the umpteenth time) – this post is about indexable IDX – my suggestion (request however I should put it) is for everyone reading this post to stay on topic and discuss the merits / negatives of spiderable IDX or don't participate.
Bob Wilson on February 26, 2010 5:56 pm
Quit whining Morgan. You deliberately drew down on a competitor and now cant take the heat because you dont control the medium.
Morgan Carey on February 26, 2010 6:10 pm
Drew – btw, I just found out (I honestly didn't know) you had taken a job with one of my competitors. I thank you for even allowing this post to stand in the first place (I thought you still worked for zillow) – but it certainly explains a lot of the response (that I was kind of surprised by) – Trust me, if I knew you were working for VR and doing work with DS (using the very technology I mentioned was the problem) I would never have agreed to do this Q & A (at least not to having it posted here on your blog).
Morgan Carey on February 26, 2010 6:12 pm
Incidentally congrats to VR on acquiring you, I am sure you will serve them very well.
Bob Wilson on February 26, 2010 6:13 pm
So why did you tweet two days ago (your 1st one in two months, btw) that their IDX Press was easily hacked?
Bob Wilson on February 26, 2010 6:15 pm
Another low blow Morgan. This was a shill post to begin with.
drewmeyers on February 26, 2010 6:15 pm
Morgan-
I assure you my new role at vr has nothing to do with the moderation
policies here. The moderation policy here has not changed in 3 years
and I would have moderated this discussion just as I did even if I
still worked for zillow.
Morgan Carey on February 26, 2010 6:17 pm
I didn't tweet Bob, I posted (on my own blog) that screenshot (that was I also posted here) and it was titled “DS Press easily hacked” – that screenshot was only put up to verify what I was asked to here in this blog – it looks as though a service called “friend feed” picked up the screenshot and tweeted it (I did not intentionally post that anywhere but on my blog, and only for the purposes of posting it here)
Morgan Carey on February 26, 2010 6:18 pm
Incidentally, after I linked the screen to here (which was my intention) I unpublished that post. Sorry, don't think you are going to find more fodder for your drama train.
Morgan Carey on February 26, 2010 6:23 pm
You win Bob
g'night
Morgan Carey on February 26, 2010 6:30 pm
I completely believe you Drew – I don't (nor do I think it came across that way) think you selectively moderated. But what I didn't realize (shame on my for not doing my homework) is that this blog is clearly a DS friendly (certainly not neutral) space (in terms of readership, not moderation policy) thus (and clearly this happened) – I was committing participation suicide by even agreeing to do this Q & A in the first place.
It's nothing against you Drew – I don't think you have wronged me here at all, and I totally have nothing against you (I was fond of you prior to this post, and I still am) – and I am a bit envious in fact – it was not a backhanded shot when I said congrats, I would have hired you in a heartbeat had I known it was an option
drewmeyers on February 27, 2010 12:41 pm
Hey Morgan-
Geek Estate strives to be a neutral place where everyone has a voice. You have an open invite to write on Geek Estate as long as you follow the author guidelines. If you're interested, email me and I'll set up an account for you.
Also, note this blog is owned by Zillow and not by me.
The Ten Best Articles in Real Estate This Week: Feb. 27, 2010 Edition on February 27, 2010 1:10 pm
[...] ) Q & A with Morgan Carey (Real Estate Webmasters) about Indexable IDX – Indexable IDX results, do they hurt you? Do they help you? If you don’t have some sort of [...]
Bob Wilson on March 4, 2010 10:16 am
Saying you were committing suicide by participating here because it is DS friendly, is patently unfair since you were the one that drew first blood.
mlbroadcast on March 6, 2010 8:17 am
This blog is completely vendor neutral and will remain so. We have new policies in place regarding the use of the blog and commenting. Please check it out. http://www.geekestateblog.com/blog-comment-policy/ – We strive to provide the RE community with quality information. Sometimes there will be debate, but it if turns into an all out competitive battle that does nothing to further the conversation, We'll take the appropriate action.
Tony on March 8, 2010 2:35 pm
I'm still hoping to hear why some MLS boards won't allow the RSS of their IDX data. No answer makes me think the boards agree with Morgan on this. Am I wrong?
atlantarealestate on March 8, 2010 9:06 pm
I can show you guys a REW powered site in Atlanta right now that is attempting to index the ENTIRE MLS in this area.
The link to the data is almost hidden in the tiny text at the bottom of the site.
66,461 addresses displayed across 887 pages.
How is this “compliant?” They are only showing the addresses. Then, the link is to a registration.
But this allows those 66k addresses to get indexed bringing traffic to that site.
Total bull. Talk about displaying only what you want, how you want.
Bob Wilson on March 8, 2010 9:14 pm
Because it is way too easy to use an rss feed to redistribute listings in ways that are not compliant.
Morgan Carey on March 9, 2010 11:58 am
Atlanta – Good catch, and in fact you are correct, if a sitemap (which is not a violation of compliance btw) is displaying any actual FMLS data (such as an address in anchor text) then the required disclaimer for FMLS should be displayed – I will report this to our IDX dept and have them address their sitemap – Can you email me the URL of the site in question? morgan @ realestatewebmasters.com
Thanks
atlantarealestate on March 9, 2010 1:19 pm
On the way.
Tnx,
Rob
gilbertperalta on March 9, 2010 8:11 pm
I guess I will come to Morgan's rescue here. I am not a customer of any company discussed here. I am still using the worse IDX ever, a non-indexable POS. I am currently in the market for an indexable IDX and a wordpress plug-in. I googled “indexable idx” and this conversation is ranked #2. Morgan was simply pointing out that DS has a product that apparently (based on his screen shot) can be hacked and non-complient. The last thing I want is to work with a company who may get in trouble and then BAM, I lose all my pages with the plug-in. It sounds like that isn't going to happen, as DS assured me they are complient.
Always be honest and speak your mind Morgan.
As for me, I'm still not sure which indexable IDX to go with….
Terra condo on March 11, 2010 11:26 pm
It is just so tempting to abandon the real estate business (at least temporarily) right now. However I’ll look into your ideas. Thanks
ericbramlett on March 12, 2010 8:13 am
I'll stay out of the RSS feed war here and just focus on the lack of logic in this statement. Silence doesn't prove or disprove anything, other than the fact that the silent entity doesn't want to respond.
If you have a moment, read up on http://www.didglennbeckrapeandmurderayounggirlin1990.com
robbspearman on March 15, 2010 9:08 pm
Morgan Carey … I would never do business with you. NEVER. Your comments and posts are unprofessional. I have decided to purchase a DS IDX site based on your behavior Morgan. The comments towards Justin and Drew are uncalled for.
REtechToday - The best articles in real estate tech for Feb 24, 2010 — REtechSource on April 27, 2010 10:14 am
[...] information help me compete with the big guns and will I get even more traffic from it? Here is a good post that answers those questions and [...]
Thomas A B Johnson on May 22, 2010 12:15 pm
Here is my take on this issue. D/S, REW matters not. Please remember that all clients of both vendors are REALTORS (otherwise they would not have MLS access.) who are bound by the REALTOR Code of Ethics. Morgan's premise that is “if can be hacked, those scummy agents will hack it” is highly offensive to me.
So the REW pitch is: “Buy our very expensive solution because your fellow REALTORS (with whom you cooperate for your livelihood) are all scumbags who will hire Estonian coders to hack your IDX feed so send us a couple grand so we can prevent it?
Morgan, please. Your product is the best premium solution available. You are better than this.
Full disclosure: I am a D/S client who should probably be a REW client if I could afford it.
Morgan Carey on May 23, 2010 11:17 am
Thomas – this thread has been dead for a while but I got an email notification, so I figure I'll respond because you said some things here that need to be addressed.
First off all, our “pitch” is buy our websites and IDX products because they are awesome and will help you generate business. Your statement about our sales pitch is quite frankly ridiculous.
As for “if it can be hacked, those scummy agents will hack it” being offensive to you. I didn't say anything about “Realtors” being scummy – I was more focusing on the fact that RSS in IDX solutions opens up the data to folks who are NOT Realtors nor bound to any COE – but since you brought it up, I hate to inform you that folks willing to cut corners, or extremely motivated to break the rules to their advantage DO exist – in fact we get thousands of inquiries per year from Realtors, who (knowing the rules full well) will make requests like “hey if we buy an IDX from you, can you make it so we hide the listing broker? I know it's required, but I don't want it – while we are at it, can we make it look like all the listings in the IDX are mine?”
Seriously – if you are one of the good guys, that's great – but don't hang me out there for posting a fact (which is commonly known) – and that is “some” Realtors are not ethical and WILL take advantage of any opportunity they can (You could replace the word “Realtors” with “people” and the same would be true – it shouldn't be something I get dumped on for, just because I point it out.
Morgan Carey on May 23, 2010 6:17 pm
Thomas – this thread has been dead for a while but I got an email notification, so I figure I'll respond because you said some things here that need to be addressed.
First off all, our “pitch” is buy our websites and IDX products because they are awesome and will help you generate business. Your statement about our sales pitch is quite frankly ridiculous.
As for “if it can be hacked, those scummy agents will hack it” being offensive to you. I didn't say anything about “Realtors” being scummy – I was more focusing on the fact that RSS in IDX solutions opens up the data to folks who are NOT Realtors nor bound to any COE – but since you brought it up, I hate to inform you that folks willing to cut corners, or extremely motivated to break the rules to their advantage DO exist – in fact we get thousands of inquiries per year from Realtors, who (knowing the rules full well) will make requests like “hey if we buy an IDX from you, can you make it so we hide the listing broker? I know it's required, but I don't want it – while we are at it, can we make it look like all the listings in the IDX are mine?”
Seriously – if you are one of the good guys, that's great – but don't hang me out there for posting a fact (which is commonly known) – and that is “some” Realtors are not ethical and WILL take advantage of any opportunity they can (You could replace the word “Realtors” with “people” and the same would be true – it shouldn't be something I get dumped on for, just because I point it out.
Do Indexed Property Pages Increase Sales? | GeekEstate Blog on July 18, 2010 5:04 pm
[...] There is a lot of buzz out there about how great it is to get property listings from your IDX on your site indexed. There are some that say it brings a great benefit for SEO and others that think it is more [...]